2024-09-21T02:38:51.086 INFO:root:teuthology version: 1.2.2.dev2+g3752d38 2024-09-21T02:38:51.120 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T02:38:51.213 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439 branch: wip-pdonnell-testing-20240920.212106-debug description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} email: pdonnell@redhat.com first_in_suite: false job_id: '7913439' kernel: client: sha1: distro kdb: 1 sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi no_nested_subset: false openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-pdonnell-testing-20240920.212106-debug ceph: cephfs: max_mds: 1 session_timeout: 300 standby_replay: true cluster-conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug mds balancer: 20 debug ms: 2 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: client mount timeout: 30 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mgr/snap_schedule/allow_m_granularity: true mgr/snap_schedule/dump_on_update: true mgr/snap_schedule/log_level: debug mon warn on pool no app: false mon: mon op complaint time: 120 osd: osd op complaint time: 180 conf: global: ms die on skipped message: false mgr: cephsqlite lock renewal timeout: 900000 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - fs.*is degraded - filesystem is degraded - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - filesystem is offline - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - insufficient standby MDS daemons available - MDS_UP_LESS_THAN_MAX - online, but wants - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - Degraded data redundancy - slow metadata IO - SLOW_OPS - slow request mgr-modules: - snap_schedule sha1: 1843221895adccfb3b692060251dd6a0183bc410 subvols: create: 2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: 1843221895adccfb3b692060251dd6a0183bc410 kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v2 selinux: allowlist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-pdonnell-testing-20240920.212106-debug sha1: 1843221895adccfb3b692060251dd6a0183bc410 owner: scheduled_pdonnell@teuthology priority: 90 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7864 sha1: 1843221895adccfb3b692060251dd6a0183bc410 sleep_before_teardown: 0 subset: 67/512 suite: fs suite_branch: wip-pdonnell-testing-20240920.212106-debug suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 1843221895adccfb3b692060251dd6a0183bc410 syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi016.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGmdOxZB6kAYWDTG9DEakmzl+2CTIqF1pt2HPcrlg+j4dewYwJpO0v5nPGIiz5nJuceUAEK+4jG1wwxLyKW86w= smithi018.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXZb8F7QPjgvayCnL5CVQbyyOspL2UbVetF/TmoblFewo3/3ST9miLooZybxD+dvd9u3+QleK7AjbR24zlsark= smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLHB9+MokhByP2d5DBChNIMg70fWQgO5XK/MSKpIXqmnMmJMpEciiDfWCRjX93e/uT1qAluhQeMHKjJgxXTuQ+o= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - kclient: null - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - workunit: clients: all: - suites/iogen.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 3752d3834a7b6cd13dc17dfaa6c2fd3658f3439a timestamp: 2024-09-20_23:48:22 tube: smithi user: pdonnell verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2902 2024-09-21T02:38:51.213 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa; will attempt to use it 2024-09-21T02:38:51.214 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks 2024-09-21T02:38:51.214 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-09-21T02:38:51.216 INFO:teuthology.task.internal:Checking packages... 2024-09-21T02:38:51.236 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '1843221895adccfb3b692060251dd6a0183bc410' 2024-09-21T02:38:51.237 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-09-21T02:38:51.237 INFO:teuthology.packaging:ref: None 2024-09-21T02:38:51.237 INFO:teuthology.packaging:tag: None 2024-09-21T02:38:51.237 INFO:teuthology.packaging:branch: wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:38:51.237 INFO:teuthology.packaging:sha1: 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:38:51.237 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-pdonnell-testing-20240920.212106-debug 2024-09-21T02:38:51.374 INFO:teuthology.task.internal:Found packages for ceph version 19.3.0-5120.g18432218 2024-09-21T02:38:51.376 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-09-21T02:38:51.382 INFO:teuthology.task.internal:no buildpackages task found 2024-09-21T02:38:51.382 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-09-21T02:38:51.405 INFO:teuthology.task.internal:Saving configuration 2024-09-21T02:38:51.427 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-09-21T02:38:51.433 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-09-21T02:38:51.459 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi016.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:34:02.523712', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJGmdOxZB6kAYWDTG9DEakmzl+2CTIqF1pt2HPcrlg+j4dewYwJpO0v5nPGIiz5nJuceUAEK+4jG1wwxLyKW86w='} 2024-09-21T02:38:51.481 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi018.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:34:02.527292', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPXZb8F7QPjgvayCnL5CVQbyyOspL2UbVetF/TmoblFewo3/3ST9miLooZybxD+dvd9u3+QleK7AjbR24zlsark='} 2024-09-21T02:38:51.501 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-09-21 02:34:02.525743', 'locked_by': 'scheduled_pdonnell@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLHB9+MokhByP2d5DBChNIMg70fWQgO5XK/MSKpIXqmnMmJMpEciiDfWCRjX93e/uT1qAluhQeMHKjJgxXTuQ+o='} 2024-09-21T02:38:51.501 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-09-21T02:38:51.507 INFO:teuthology.task.internal:roles: ubuntu@smithi016.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-09-21T02:38:51.507 INFO:teuthology.task.internal:roles: ubuntu@smithi018.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-09-21T02:38:51.507 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-09-21T02:38:51.507 INFO:teuthology.run_tasks:Running task console_log... 2024-09-21T02:38:51.605 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fb6cacf35b0>, signals=[15]) 2024-09-21T02:38:51.605 INFO:teuthology.run_tasks:Running task internal.connect... 2024-09-21T02:38:51.611 INFO:teuthology.task.internal:Opening connections... 2024-09-21T02:38:51.611 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T02:38:51.612 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:38:51.689 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi018.front.sepia.ceph.com 2024-09-21T02:38:51.690 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:38:51.763 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2024-09-21T02:38:51.764 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:38:51.838 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-09-21T02:38:51.845 DEBUG:teuthology.orchestra.run.smithi016:> uname -m 2024-09-21T02:38:51.863 INFO:teuthology.orchestra.run.smithi016.stdout:x86_64 2024-09-21T02:38:51.863 DEBUG:teuthology.orchestra.run.smithi016:> cat /etc/os-release 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:NAME="CentOS Stream" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION="9" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:ID="centos" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:VERSION_ID="9" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:ANSI_COLOR="0;31" 2024-09-21T02:38:51.920 INFO:teuthology.orchestra.run.smithi016.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:38:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:38:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:38:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:38:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:38:51.921 INFO:teuthology.orchestra.run.smithi016.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:38:51.921 INFO:teuthology.lock.ops:Updating smithi016.front.sepia.ceph.com on lock server 2024-09-21T02:38:51.939 DEBUG:teuthology.orchestra.run.smithi018:> uname -m 2024-09-21T02:38:51.957 INFO:teuthology.orchestra.run.smithi018.stdout:x86_64 2024-09-21T02:38:51.957 DEBUG:teuthology.orchestra.run.smithi018:> cat /etc/os-release 2024-09-21T02:38:52.015 INFO:teuthology.orchestra.run.smithi018.stdout:NAME="CentOS Stream" 2024-09-21T02:38:52.015 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION="9" 2024-09-21T02:38:52.015 INFO:teuthology.orchestra.run.smithi018.stdout:ID="centos" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:VERSION_ID="9" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:ANSI_COLOR="0;31" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:38:52.016 INFO:teuthology.orchestra.run.smithi018.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:38:52.017 INFO:teuthology.lock.ops:Updating smithi018.front.sepia.ceph.com on lock server 2024-09-21T02:38:52.040 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2024-09-21T02:38:52.058 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2024-09-21T02:38:52.058 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2024-09-21T02:38:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2024-09-21T02:38:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="9" 2024-09-21T02:38:52.116 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="9" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el9" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 9" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:LOGO="fedora-logo-icon" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-09-21T02:38:52.117 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-09-21T02:38:52.117 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2024-09-21T02:38:52.139 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-09-21T02:38:52.149 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-09-21T02:38:52.180 INFO:teuthology.task.internal:Checking for old test directory... 2024-09-21T02:38:52.181 DEBUG:teuthology.orchestra.run.smithi016:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:38:52.183 DEBUG:teuthology.orchestra.run.smithi018:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:38:52.185 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2024-09-21T02:38:52.202 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-09-21T02:38:52.214 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-09-21T02:38:52.214 DEBUG:teuthology.orchestra.run.smithi016:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:38:52.240 DEBUG:teuthology.orchestra.run.smithi018:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:38:52.242 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2024-09-21T02:38:52.270 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-09-21T02:38:52.341 INFO:teuthology.run_tasks:Running task kernel... 2024-09-21T02:38:52.355 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': 1, 'sha1': 'distro'} 2024-09-21T02:38:52.355 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-09-21T02:38:52.355 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-09-21T02:38:52.355 DEBUG:teuthology.orchestra.run.smithi016:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:38:52.372 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:38:52.372 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-21T02:38:52.430 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-503.el9.x86_64 2024-09-21T02:38:52.430 INFO:teuthology.task.kernel:Running kernel on smithi016: 5.14.0-503.el9.x86_64 2024-09-21T02:38:52.431 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum install -y kernel 2024-09-21T02:38:55.885 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 5.2 MB/s | 8.3 MB 00:01 2024-09-21T02:39:11.877 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 1.4 MB/s | 20 MB 00:14 2024-09-21T02:39:18.552 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 6.5 MB 00:02 2024-09-21T02:39:20.278 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 59 kB/s | 19 kB 00:00 2024-09-21T02:39:21.065 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 34 MB/s | 23 MB 00:00 2024-09-21T02:39:26.863 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 30 kB/s | 1.7 kB 00:00 2024-09-21T02:39:28.844 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-09-21T02:39:28.844 INFO:teuthology.orchestra.run.smithi016.stdout:Package kernel-5.14.0-503.el9.x86_64 is already installed. 2024-09-21T02:39:28.943 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-21T02:39:28.952 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T02:39:28.952 INFO:teuthology.orchestra.run.smithi016.stdout: Package Architecture Version Repository Size 2024-09-21T02:39:28.952 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T02:39:28.952 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-21T02:39:28.952 INFO:teuthology.orchestra.run.smithi016.stdout: kernel x86_64 5.14.0-508.el9 baseos 36 k 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core x86_64 5.14.0-508.el9 baseos 16 M 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules x86_64 5.14.0-508.el9 baseos 35 M 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core x86_64 5.14.0-508.el9 baseos 29 M 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout:================================================================================ 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout:Install 4 Packages 2024-09-21T02:39:28.953 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:39:28.954 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 79 M 2024-09-21T02:39:28.954 INFO:teuthology.orchestra.run.smithi016.stdout:Installed size: 126 M 2024-09-21T02:39:28.954 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-21T02:39:29.340 INFO:teuthology.orchestra.run.smithi016.stdout:(1/4): kernel-5.14.0-508.el9.x86_64.rpm 127 kB/s | 36 kB 00:00 2024-09-21T02:39:37.781 INFO:teuthology.orchestra.run.smithi016.stdout:(2/4): kernel-core-5.14.0-508.el9.x86_64.rpm 1.8 MB/s | 16 MB 00:08 2024-09-21T02:39:42.063 INFO:teuthology.orchestra.run.smithi016.stdout:(3/4): kernel-modules-5.14.0-508.el9.x86_64.rpm 2.7 MB/s | 35 MB 00:13 2024-09-21T02:40:03.187 INFO:teuthology.orchestra.run.smithi016.stdout:(4/4): kernel-modules-core-5.14.0-508.el9.x86_6 865 kB/s | 29 MB 00:33 2024-09-21T02:40:03.187 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:40:03.187 INFO:teuthology.orchestra.run.smithi016.stdout:Total 2.3 MB/s | 79 MB 00:34 2024-09-21T02:40:03.900 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-21T02:40:04.304 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-21T02:40:04.304 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-21T02:40:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-21T02:40:05.262 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-21T02:40:07.829 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-21T02:40:08.483 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-core-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:40:08.563 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:40:11.318 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:40:11.466 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:40:16.594 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:40:16.633 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:40:21.776 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:41:01.215 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:41:01.256 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-modules-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:41:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: kernel-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:41:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-5.14.0-508.el9.x86_64 1/4 2024-09-21T02:41:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-core-5.14.0-508.el9.x86_64 2/4 2024-09-21T02:41:01.615 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-5.14.0-508.el9.x86_64 3/4 2024-09-21T02:41:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : kernel-modules-core-5.14.0-508.el9.x86_64 4/4 2024-09-21T02:41:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:41:02.502 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-21T02:41:02.502 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-5.14.0-508.el9.x86_64 2024-09-21T02:41:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-core-5.14.0-508.el9.x86_64 2024-09-21T02:41:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-5.14.0-508.el9.x86_64 2024-09-21T02:41:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: kernel-modules-core-5.14.0-508.el9.x86_64 2024-09-21T02:41:02.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:41:02.503 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-21T02:41:02.866 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:41:02.967 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:41:02.968 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:41:02.968 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-503.el9.x86_64 Expected: 5.14.0-508.el9.x86_64 2024-09-21T02:41:02.968 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2024-09-21T02:41:02.968 DEBUG:teuthology.task.kernel:src is distro, skipping download 2024-09-21T02:41:02.968 INFO:teuthology.task.kernel:Installing distro kernel on client.0... 2024-09-21T02:41:02.968 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi016.front.sepia.ceph.com, path=None, version=distro) 2024-09-21T02:41:02.968 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q kernel | sort -rV | head -n 1 2024-09-21T02:41:03.027 INFO:teuthology.orchestra.run.smithi016.stdout:kernel-5.14.0-508.el9.x86_64 2024-09-21T02:41:03.028 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-508.el9.x86_64 2024-09-21T02:41:03.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo rpm -qi grub2-tools 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Name : grub2-tools 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Epoch : 1 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Version : 2.06 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Release : 82.el9 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Architecture: x86_64 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Install Date: Wed 28 Aug 2024 09:06:27 PM UTC 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Group : Unspecified 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:Size : 8274670 2024-09-21T02:41:03.061 INFO:teuthology.orchestra.run.smithi016.stdout:License : GPLv3+ 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Signature : RSA/SHA256, Mon 01 Jul 2024 07:46:42 AM UTC, Key ID 05b555b38483c65d 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Source RPM : grub2-2.06-82.el9.src.rpm 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Build Date : Fri 28 Jun 2024 09:24:44 AM UTC 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Packager : builder@centos.org 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Vendor : CentOS 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:URL : http://www.gnu.org/software/grub/ 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Summary : Support tools for GRUB. 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:Description : 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2024-09-21T02:41:03.062 INFO:teuthology.orchestra.run.smithi016.stdout:customizable bootloader with modular architecture. It supports a rich 2024-09-21T02:41:03.063 INFO:teuthology.orchestra.run.smithi016.stdout:variety of kernel formats, file systems, computer architectures and 2024-09-21T02:41:03.063 INFO:teuthology.orchestra.run.smithi016.stdout:hardware devices. 2024-09-21T02:41:03.063 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:41:03.063 INFO:teuthology.orchestra.run.smithi016.stdout:This subpackage provides tools for support of all platforms. 2024-09-21T02:41:03.064 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2024-09-21T02:41:03.064 INFO:teuthology.task.kernel:Updating grub on smithi016 to boot 5.14.0-508.el9.x86_64 2024-09-21T02:41:03.064 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2024-09-21T02:41:03.786 INFO:teuthology.orchestra.run.smithi016.stderr:Generating grub configuration file ... 2024-09-21T02:41:05.097 INFO:teuthology.orchestra.run.smithi016.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2024-09-21T02:41:05.138 INFO:teuthology.orchestra.run.smithi016.stderr:done 2024-09-21T02:41:05.141 DEBUG:teuthology.orchestra.run.smithi016:> mktemp 2024-09-21T02:41:05.156 INFO:teuthology.orchestra.run.smithi016.stdout:/tmp/tmp.bzJ3z2gpsI 2024-09-21T02:41:05.156 DEBUG:teuthology.orchestra.run.smithi016:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.bzJ3z2gpsI 2024-09-21T02:41:05.222 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0666 /tmp/tmp.bzJ3z2gpsI 2024-09-21T02:41:05.370 DEBUG:teuthology.orchestra.remote:smithi016:/tmp/tmp.bzJ3z2gpsI is 6KB 2024-09-21T02:41:05.389 DEBUG:teuthology.orchestra.run.smithi016:> rm -fr /tmp/tmp.bzJ3z2gpsI 2024-09-21T02:41:05.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /bin/ls /boot/loader/entries || true 2024-09-21T02:41:05.466 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-0-rescue.conf 2024-09-21T02:41:05.466 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-239.el9.x86_64.conf 2024-09-21T02:41:05.466 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-503.el9.x86_64.conf 2024-09-21T02:41:05.466 INFO:teuthology.orchestra.run.smithi016.stdout:fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64.conf 2024-09-21T02:41:05.467 DEBUG:teuthology.orchestra.run.smithi016:> sudo grub2-set-default fbf6b6fa6cf04654a0e563f30ecbf43a-5.14.0-508.el9.x86_64 2024-09-21T02:41:05.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo shutdown -r now 2024-09-21T02:41:35.620 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2024-09-21T02:41:35.621 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T02:41:35.621 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:42:35.623 DEBUG:teuthology.orchestra.remote:timed out 2024-09-21T02:42:44.623 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T02:42:44.624 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:42:44.959 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-21T02:42:45.551 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T02:42:45.551 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "5.14.0-508.el9.x86_64"... 2024-09-21T02:42:45.552 DEBUG:teuthology.orchestra.run.smithi016:> uname -r 2024-09-21T02:42:45.567 INFO:teuthology.orchestra.run.smithi016.stdout:5.14.0-508.el9.x86_64 2024-09-21T02:42:45.567 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-508.el9.x86_64 vs 5.14.0-508.el9.x86_64 2024-09-21T02:42:45.567 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2024-09-21T02:42:45.567 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2024-09-21T02:42:46.568 DEBUG:teuthology.parallel:result is None 2024-09-21T02:42:46.568 INFO:teuthology.run_tasks:Running task internal.base... 2024-09-21T02:42:46.577 INFO:teuthology.task.internal:Creating test directory... 2024-09-21T02:42:46.577 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:42:46.579 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:42:46.582 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-09-21T02:42:46.601 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-09-21T02:42:46.608 INFO:teuthology.run_tasks:Running task internal.archive... 2024-09-21T02:42:46.628 INFO:teuthology.task.internal:Creating archive directory... 2024-09-21T02:42:46.628 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:42:46.635 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:42:46.641 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-09-21T02:42:46.682 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-09-21T02:42:46.688 INFO:teuthology.task.internal:Enabling coredump saving... 2024-09-21T02:42:46.688 DEBUG:teuthology.orchestra.run.smithi018:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:42:46.726 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:42:46.726 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2024-09-21T02:42:46.742 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:42:46.743 DEBUG:teuthology.orchestra.run.smithi016:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:42:46.745 DEBUG:teuthology.orchestra.run.smithi018:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:42:46.769 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-09-21T02:42:46.826 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.829 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.841 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.847 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.864 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.893 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-09-21T02:42:46.895 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-09-21T02:42:46.901 INFO:teuthology.task.internal:Configuring sudo... 2024-09-21T02:42:46.902 DEBUG:teuthology.orchestra.run.smithi016:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:42:46.904 DEBUG:teuthology.orchestra.run.smithi018:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:42:46.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-09-21T02:42:46.933 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-09-21T02:42:46.963 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-09-21T02:42:46.964 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:42:46.969 DEBUG:teuthology.orchestra.run.smithi018:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:42:46.974 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-09-21T02:42:46.991 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.124 DEBUG:teuthology.orchestra.run.smithi016:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.166 DEBUG:teuthology.orchestra.run.smithi016:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.206 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:42:47.206 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:42:47.279 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.324 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.376 DEBUG:teuthology.orchestra.run.smithi018:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.461 DEBUG:teuthology.orchestra.run.smithi018:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.503 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:42:47.503 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:42:47.571 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.617 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-09-21T02:42:47.683 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-09-21T02:42:47.769 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:42:47.769 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-09-21T02:42:47.835 DEBUG:teuthology.orchestra.run.smithi016:> sudo service rsyslog restart 2024-09-21T02:42:47.837 DEBUG:teuthology.orchestra.run.smithi018:> sudo service rsyslog restart 2024-09-21T02:42:47.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2024-09-21T02:42:47.878 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:42:47.892 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:42:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T02:42:48.231 INFO:teuthology.run_tasks:Running task internal.timer... 2024-09-21T02:42:48.271 INFO:teuthology.task.internal:Starting timer... 2024-09-21T02:42:48.271 INFO:teuthology.run_tasks:Running task pcp... 2024-09-21T02:42:48.296 INFO:teuthology.run_tasks:Running task selinux... 2024-09-21T02:42:48.317 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:logrotate_t:s0']} 2024-09-21T02:42:48.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo service auditd rotate 2024-09-21T02:42:48.404 INFO:teuthology.orchestra.run.smithi016.stdout:Rotating logs: 2024-09-21T02:42:48.405 DEBUG:teuthology.orchestra.run.smithi018:> sudo service auditd rotate 2024-09-21T02:42:48.479 INFO:teuthology.orchestra.run.smithi018.stdout:Rotating logs: 2024-09-21T02:42:48.481 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2024-09-21T02:42:48.551 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2024-09-21T02:42:48.553 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-09-21T02:42:48.553 DEBUG:teuthology.orchestra.run.smithi016:> /usr/sbin/getenforce 2024-09-21T02:42:48.592 INFO:teuthology.orchestra.run.smithi016.stdout:Permissive 2024-09-21T02:42:48.593 DEBUG:teuthology.orchestra.run.smithi018:> /usr/sbin/getenforce 2024-09-21T02:42:48.627 INFO:teuthology.orchestra.run.smithi018.stdout:Permissive 2024-09-21T02:42:48.627 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2024-09-21T02:42:48.655 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2024-09-21T02:42:48.655 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi016.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi018.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi123.front.sepia.ceph.com': 'permissive'} 2024-09-21T02:42:48.656 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:42:48.684 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:42:48.684 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:42:48.728 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:42:48.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T02:42:48.756 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T02:42:48.757 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-09-21T02:42:48.757 DEBUG:teuthology.orchestra.run.smithi016:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:42:48.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:42:48.834 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2024-09-21T02:42:48.866 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-09-21T02:42:48.884 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-09-21T02:42:48.886 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-09-21T02:42:49.003 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-09-21T02:42:49.012 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-09-21T02:42:49.013 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi016.front.sepia.ceph.com,smithi018.front.sepia.ceph.com,smithi123.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-09-21T02:46:51.940 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi016.front.sepia.ceph.com'), Remote(name='ubuntu@smithi018.front.sepia.ceph.com'), Remote(name='ubuntu@smithi123.front.sepia.ceph.com')] 2024-09-21T02:46:51.941 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T02:46:51.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi016.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:46:52.017 DEBUG:teuthology.orchestra.run.smithi016:> true 2024-09-21T02:46:52.099 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi016.front.sepia.ceph.com' 2024-09-21T02:46:52.100 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-21T02:46:52.100 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi018.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:46:52.174 DEBUG:teuthology.orchestra.run.smithi018:> true 2024-09-21T02:46:52.249 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi018.front.sepia.ceph.com' 2024-09-21T02:46:52.250 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-21T02:46:52.250 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-09-21T02:46:52.326 DEBUG:teuthology.orchestra.run.smithi123:> true 2024-09-21T02:46:52.411 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2024-09-21T02:46:52.411 INFO:teuthology.run_tasks:Running task clock... 2024-09-21T02:46:52.420 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-09-21T02:46:52.420 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:46:52.420 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:46:52.422 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:46:52.422 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:46:52.425 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-09-21T02:46:52.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T02:46:52.453 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:46:52.454 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:46:52.468 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:46:52.470 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:46:52.481 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-09-21T02:46:52.493 INFO:teuthology.orchestra.run.smithi018.stderr:sudo: ntpd: command not found 2024-09-21T02:46:52.496 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-09-21T02:46:52.497 INFO:teuthology.orchestra.run.smithi016.stderr:sudo: ntpd: command not found 2024-09-21T02:46:52.504 INFO:teuthology.orchestra.run.smithi018.stdout:506 Cannot talk to daemon 2024-09-21T02:46:52.509 INFO:teuthology.orchestra.run.smithi016.stdout:506 Cannot talk to daemon 2024-09-21T02:46:52.519 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:46:52.521 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2024-09-21T02:46:52.524 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:46:52.532 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2024-09-21T02:46:52.533 INFO:teuthology.orchestra.run.smithi018.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:46:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:46:52.546 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-09-21T02:46:52.561 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-09-21T02:46:52.594 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:46:52.597 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:46:52.598 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-21T02:46:52.598 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.598 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.598 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.598 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.599 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.603 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.616 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.621 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T02:46:52.621 INFO:teuthology.run_tasks:Running task install... 2024-09-21T02:46:52.640 DEBUG:teuthology.task.install:project ceph 2024-09-21T02:46:52.640 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'}} 2024-09-21T02:46:52.641 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410'} 2024-09-21T02:46:52.641 INFO:teuthology.task.install:Using flavor: default 2024-09-21T02:46:52.646 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-09-21T02:46:52.646 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-09-21T02:46:52.646 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:46:52.647 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:46:52.648 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:46:52.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:46:52.649 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'tag': None, 'wait_for_package': False} 2024-09-21T02:46:52.649 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:46:52.799 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:46:52.799 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:46:52.802 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:46:52.802 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:46:52.806 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/ 2024-09-21T02:46:52.806 INFO:teuthology.task.install.rpm:Package version is 19.3.0-5120.g18432218 2024-09-21T02:46:52.905 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:46:52.905 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:46:52.905 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:46:52.911 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:46:52.911 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:46:52.912 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:46:52.932 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:46:52.932 DEBUG:teuthology.orchestra.run.smithi016:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:46:52.936 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-09-21T02:46:52.936 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:46:52.936 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-09-21T02:46:52.937 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:46:52.937 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:46:52.962 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-09-21T02:46:52.962 DEBUG:teuthology.orchestra.run.smithi018:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/1843221895adccfb3b692060251dd6a0183bc410/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-09-21T02:46:52.999 DEBUG:teuthology.orchestra.run.smithi016:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:46:53.003 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:46:53.028 DEBUG:teuthology.orchestra.run.smithi018:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-09-21T02:46:53.075 DEBUG:teuthology.orchestra.run.smithi016:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:46:53.077 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:46:53.104 DEBUG:teuthology.orchestra.run.smithi018:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-09-21T02:46:53.140 INFO:teuthology.orchestra.run.smithi016.stdout:check_obsoletes = 1 2024-09-21T02:46:53.142 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum clean all 2024-09-21T02:46:53.142 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2024-09-21T02:46:53.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2024-09-21T02:46:53.168 INFO:teuthology.orchestra.run.smithi018.stdout:check_obsoletes = 1 2024-09-21T02:46:53.170 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum clean all 2024-09-21T02:46:53.365 INFO:teuthology.orchestra.run.smithi123.stdout:45 files removed 2024-09-21T02:46:53.381 INFO:teuthology.orchestra.run.smithi016.stdout:45 files removed 2024-09-21T02:46:53.392 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:46:53.400 INFO:teuthology.orchestra.run.smithi018.stdout:45 files removed 2024-09-21T02:46:53.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:46:53.428 DEBUG:teuthology.orchestra.run.smithi018:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-09-21T02:46:53.935 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 314 kB/s | 84 kB 00:00 2024-09-21T02:46:53.980 INFO:teuthology.orchestra.run.smithi018.stdout:ceph packages for x86_64 312 kB/s | 84 kB 00:00 2024-09-21T02:46:53.981 INFO:teuthology.orchestra.run.smithi016.stdout:ceph packages for x86_64 308 kB/s | 84 kB 00:00 2024-09-21T02:46:54.228 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 97 kB/s | 24 kB 00:00 2024-09-21T02:46:54.264 INFO:teuthology.orchestra.run.smithi018.stdout:ceph noarch packages 101 kB/s | 24 kB 00:00 2024-09-21T02:46:54.282 INFO:teuthology.orchestra.run.smithi016.stdout:ceph noarch packages 95 kB/s | 24 kB 00:00 2024-09-21T02:46:54.511 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 7.7 kB/s | 1.9 kB 00:00 2024-09-21T02:46:54.539 INFO:teuthology.orchestra.run.smithi018.stdout:ceph source packages 8.0 kB/s | 1.9 kB 00:00 2024-09-21T02:46:54.566 INFO:teuthology.orchestra.run.smithi016.stdout:ceph source packages 7.8 kB/s | 1.9 kB 00:00 2024-09-21T02:46:54.948 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - BaseOS 22 MB/s | 8.3 MB 00:00 2024-09-21T02:46:55.205 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.3 MB 00:00 2024-09-21T02:46:56.274 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - BaseOS 4.9 MB/s | 8.3 MB 00:01 2024-09-21T02:46:57.221 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - AppStream 21 MB/s | 20 MB 00:00 2024-09-21T02:47:00.861 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - AppStream 5.9 MB/s | 20 MB 00:03 2024-09-21T02:47:03.766 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - CRB 2.3 MB/s | 6.5 MB 00:02 2024-09-21T02:47:03.824 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - AppStream 2.6 MB/s | 20 MB 00:07 2024-09-21T02:47:05.214 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 9 - Extras packages 80 kB/s | 19 kB 00:00 2024-09-21T02:47:05.462 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - CRB 10 MB/s | 6.5 MB 00:00 2024-09-21T02:47:06.060 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 31 MB/s | 23 MB 00:00 2024-09-21T02:47:07.296 INFO:teuthology.orchestra.run.smithi016.stdout:CentOS Stream 9 - Extras packages 34 kB/s | 19 kB 00:00 2024-09-21T02:47:08.224 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - CRB 14 MB/s | 6.5 MB 00:00 2024-09-21T02:47:08.451 INFO:teuthology.orchestra.run.smithi016.stdout:Extra Packages for Enterprise Linux 22 MB/s | 23 MB 00:01 2024-09-21T02:47:09.800 INFO:teuthology.orchestra.run.smithi018.stdout:CentOS Stream 9 - Extras packages 99 kB/s | 19 kB 00:00 2024-09-21T02:47:10.645 INFO:teuthology.orchestra.run.smithi018.stdout:Extra Packages for Enterprise Linux 32 MB/s | 23 MB 00:00 2024-09-21T02:47:11.499 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 47 kB/s | 1.7 kB 00:00 2024-09-21T02:47:13.643 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:13.644 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:13.657 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:47:13.658 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:47:13.659 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:47:13.660 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:47:13.661 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:47:13.664 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:47:13.665 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:47:13.728 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:47:13.754 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:47:13.755 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:47:13.756 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:47:13.757 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:47:13.758 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:47:13.759 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:47:13.760 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:47:13.761 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:47:13.762 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:47:13.763 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:47:13.764 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout:Installing weak dependencies: 2024-09-21T02:47:13.765 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:======================================================================================= 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:Install 149 Packages 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 8 Packages 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 228 M 2024-09-21T02:47:13.766 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2024-09-21T02:47:14.015 INFO:teuthology.orchestra.run.smithi016.stdout:lab-extras 31 kB/s | 1.7 kB 00:00 2024-09-21T02:47:14.075 INFO:teuthology.orchestra.run.smithi123.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 57 kB/s | 6.6 kB 00:00 2024-09-21T02:47:14.317 INFO:teuthology.orchestra.run.smithi123.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.5 MB/s | 873 kB 00:00 2024-09-21T02:47:14.401 INFO:teuthology.orchestra.run.smithi123.stdout:(3/157): ceph-base-19.3.0-5120.g18432218.el9.x8 13 MB/s | 5.5 MB 00:00 2024-09-21T02:47:14.426 INFO:teuthology.orchestra.run.smithi123.stdout:(4/157): ceph-immutable-object-cache-19.3.0-512 1.3 MB/s | 148 kB 00:00 2024-09-21T02:47:14.510 INFO:teuthology.orchestra.run.smithi123.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 21 MB/s | 2.3 MB 00:00 2024-09-21T02:47:14.568 INFO:teuthology.orchestra.run.smithi123.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 11 MB/s | 1.5 MB 00:00 2024-09-21T02:47:14.785 INFO:teuthology.orchestra.run.smithi123.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 18 MB/s | 5.0 MB 00:00 2024-09-21T02:47:15.127 INFO:teuthology.orchestra.run.smithi123.stdout:(8/157): ceph-common-19.3.0-5120.g18432218.el9. 17 MB/s | 20 MB 00:01 2024-09-21T02:47:15.202 INFO:teuthology.orchestra.run.smithi123.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 337 kB/s | 25 kB 00:00 2024-09-21T02:47:15.394 INFO:teuthology.orchestra.run.smithi123.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 17 MB/s | 10 MB 00:00 2024-09-21T02:47:15.461 INFO:teuthology.orchestra.run.smithi123.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 3.3 MB/s | 226 kB 00:00 2024-09-21T02:47:15.628 INFO:teuthology.orchestra.run.smithi123.stdout:(12/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 17 MB/s | 18 MB 00:01 2024-09-21T02:47:15.653 INFO:teuthology.orchestra.run.smithi123.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 141 kB/s | 27 kB 00:00 2024-09-21T02:47:15.679 INFO:teuthology.orchestra.run.smithi123.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 636 kB/s | 32 kB 00:00 2024-09-21T02:47:15.721 INFO:teuthology.orchestra.run.smithi123.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 11 MB/s | 766 kB 00:00 2024-09-21T02:47:15.746 INFO:teuthology.orchestra.run.smithi123.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.3 MB/s | 158 kB 00:00 2024-09-21T02:47:15.771 INFO:teuthology.orchestra.run.smithi123.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.5 MB/s | 127 kB 00:00 2024-09-21T02:47:15.805 INFO:teuthology.orchestra.run.smithi123.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 8.5 MB/s | 507 kB 00:00 2024-09-21T02:47:15.863 INFO:teuthology.orchestra.run.smithi123.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 775 kB/s | 45 kB 00:00 2024-09-21T02:47:15.947 INFO:teuthology.orchestra.run.smithi123.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.7 MB/s | 148 kB 00:00 2024-09-21T02:47:16.038 INFO:teuthology.orchestra.run.smithi123.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 20 MB/s | 5.3 MB 00:00 2024-09-21T02:47:16.072 INFO:teuthology.orchestra.run.smithi123.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T02:47:16.097 INFO:teuthology.orchestra.run.smithi123.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 5.4 MB/s | 321 kB 00:00 2024-09-21T02:47:16.123 INFO:teuthology.orchestra.run.smithi123.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T02:47:16.148 INFO:teuthology.orchestra.run.smithi123.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T02:47:16.173 INFO:teuthology.orchestra.run.smithi123.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T02:47:16.202 INFO:teuthology.orchestra.run.smithi018.stdout:lab-extras 38 kB/s | 1.7 kB 00:00 2024-09-21T02:47:16.232 INFO:teuthology.orchestra.run.smithi123.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.9 MB/s | 170 kB 00:00 2024-09-21T02:47:16.299 INFO:teuthology.orchestra.run.smithi123.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 21 MB/s | 3.1 MB 00:00 2024-09-21T02:47:16.311 INFO:teuthology.orchestra.run.smithi016.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:16.312 INFO:teuthology.orchestra.run.smithi016.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:16.316 INFO:teuthology.orchestra.run.smithi123.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 410 kB/s | 34 kB 00:00 2024-09-21T02:47:16.326 INFO:teuthology.orchestra.run.smithi016.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:47:16.327 INFO:teuthology.orchestra.run.smithi016.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:47:16.328 INFO:teuthology.orchestra.run.smithi016.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:47:16.329 INFO:teuthology.orchestra.run.smithi016.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:47:16.331 INFO:teuthology.orchestra.run.smithi016.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:47:16.334 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:47:16.335 INFO:teuthology.orchestra.run.smithi016.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:47:16.349 INFO:teuthology.orchestra.run.smithi123.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:47:16.402 INFO:teuthology.orchestra.run.smithi016.stdout:Dependencies resolved. 2024-09-21T02:47:16.419 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T02:47:16.419 INFO:teuthology.orchestra.run.smithi016.stdout: Package Arch Version Repository Size 2024-09-21T02:47:16.419 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T02:47:16.419 INFO:teuthology.orchestra.run.smithi016.stdout:Installing: 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:47:16.420 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:47:16.421 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrading: 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:47:16.422 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout:Installing dependencies: 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:47:16.423 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:47:16.424 INFO:teuthology.orchestra.run.smithi016.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:47:16.425 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:16.426 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:47:16.427 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:47:16.428 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:47:16.429 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:47:16.430 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:Installing weak dependencies: 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction Summary 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:======================================================================================= 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:Install 149 Packages 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout:Upgrade 8 Packages 2024-09-21T02:47:16.431 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:47:16.432 INFO:teuthology.orchestra.run.smithi016.stdout:Total download size: 228 M 2024-09-21T02:47:16.432 INFO:teuthology.orchestra.run.smithi016.stdout:Downloading Packages: 2024-09-21T02:47:16.808 INFO:teuthology.orchestra.run.smithi123.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 16 MB/s | 7.4 MB 00:00 2024-09-21T02:47:16.891 INFO:teuthology.orchestra.run.smithi123.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.4 MB/s | 292 kB 00:00 2024-09-21T02:47:16.967 INFO:teuthology.orchestra.run.smithi123.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 659 kB/s | 49 kB 00:00 2024-09-21T02:47:17.050 INFO:teuthology.orchestra.run.smithi016.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 42 kB/s | 6.6 kB 00:00 2024-09-21T02:47:17.109 INFO:teuthology.orchestra.run.smithi123.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 18 MB/s | 14 MB 00:00 2024-09-21T02:47:17.134 INFO:teuthology.orchestra.run.smithi123.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 99 kB/s | 16 kB 00:00 2024-09-21T02:47:17.168 INFO:teuthology.orchestra.run.smithi123.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 4.8 MB/s | 285 kB 00:00 2024-09-21T02:47:17.284 INFO:teuthology.orchestra.run.smithi016.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 3.7 MB/s | 873 kB 00:00 2024-09-21T02:47:17.367 INFO:teuthology.orchestra.run.smithi016.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 1.7 MB/s | 148 kB 00:00 2024-09-21T02:47:17.451 INFO:teuthology.orchestra.run.smithi016.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 9.9 MB/s | 5.5 MB 00:00 2024-09-21T02:47:17.526 INFO:teuthology.orchestra.run.smithi123.stdout:(37/157): ceph-test-19.3.0-5120.g18432218.el9.x 17 MB/s | 40 MB 00:02 2024-09-21T02:47:17.543 INFO:teuthology.orchestra.run.smithi016.stdout:(5/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 17 MB/s | 1.5 MB 00:00 2024-09-21T02:47:17.551 INFO:teuthology.orchestra.run.smithi123.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 48 kB/s | 18 kB 00:00 2024-09-21T02:47:17.601 INFO:teuthology.orchestra.run.smithi016.stdout:(6/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 9.8 MB/s | 2.3 MB 00:00 2024-09-21T02:47:17.610 INFO:teuthology.orchestra.run.smithi123.stdout:(39/157): cephadm-19.3.0-5120.g18432218.el9.noa 1.6 MB/s | 784 kB 00:00 2024-09-21T02:47:18.019 INFO:teuthology.orchestra.run.smithi016.stdout:(7/157): ceph-common-19.3.0-5120.g18432218.el9. 18 MB/s | 20 MB 00:01 2024-09-21T02:47:18.019 INFO:teuthology.orchestra.run.smithi123.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 162 kB/s | 80 kB 00:00 2024-09-21T02:47:18.086 INFO:teuthology.orchestra.run.smithi016.stdout:(8/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 9.2 MB/s | 5.0 MB 00:00 2024-09-21T02:47:18.144 INFO:teuthology.orchestra.run.smithi016.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 433 kB/s | 25 kB 00:00 2024-09-21T02:47:18.244 INFO:teuthology.orchestra.run.smithi123.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 302 kB/s | 191 kB 00:00 2024-09-21T02:47:18.311 INFO:teuthology.orchestra.run.smithi123.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 867 kB/s | 253 kB 00:00 2024-09-21T02:47:18.361 INFO:teuthology.orchestra.run.smithi123.stdout:(43/157): libgfortran-11.5.0-2.el9.x86_64.rpm 990 kB/s | 801 kB 00:00 2024-09-21T02:47:18.420 INFO:teuthology.orchestra.run.smithi123.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 981 kB/s | 106 kB 00:00 2024-09-21T02:47:18.438 INFO:teuthology.orchestra.run.smithi018.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:18.438 INFO:teuthology.orchestra.run.smithi018.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-09-21T02:47:18.450 INFO:teuthology.orchestra.run.smithi018.stdout:Package NetworkManager-1:1.48.8-1.el9.x86_64 is already installed. 2024-09-21T02:47:18.451 INFO:teuthology.orchestra.run.smithi018.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-09-21T02:47:18.452 INFO:teuthology.orchestra.run.smithi018.stdout:Package util-linux-2.37.4-20.el9.x86_64 is already installed. 2024-09-21T02:47:18.453 INFO:teuthology.orchestra.run.smithi018.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-09-21T02:47:18.454 INFO:teuthology.orchestra.run.smithi018.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-09-21T02:47:18.457 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-09-21T02:47:18.458 INFO:teuthology.orchestra.run.smithi018.stdout:Package xfsprogs-6.4.0-4.el9.x86_64 is already installed. 2024-09-21T02:47:18.478 INFO:teuthology.orchestra.run.smithi123.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.1 MB/s | 135 kB 00:00 2024-09-21T02:47:18.523 INFO:teuthology.orchestra.run.smithi018.stdout:Dependencies resolved. 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout: Package Arch Version Repository Size 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout:Installing: 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout: ceph x86_64 2:19.3.0-5120.g18432218.el9 ceph 6.6 k 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.5 M 2024-09-21T02:47:18.536 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 873 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr x86_64 2:19.3.0-5120.g18432218.el9 ceph 1.5 M 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 154 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 14 M 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 7.4 M 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 49 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 10 M 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test x86_64 2:19.3.0-5120.g18432218.el9 ceph 40 M 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 285 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 784 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 226 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell x86_64 2:19.3.0-5120.g18432218.el9 ceph 27 k 2024-09-21T02:47:18.537 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 18 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel x86_64 0.191-4.el9 appstream 22 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 32 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 766 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel x86_64 2:19.3.0-5120.g18432218.el9 ceph 127 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: libtool x86_64 2.4.6-46.el9 appstream 581 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel x86_64 2.37.4-20.el9 appstream 24 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel x86_64 1:3.2.2-6.el9 appstream 4.4 M 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql x86_64 13.16-1.el9 appstream 1.6 M 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib x86_64 13.16-1.el9 appstream 910 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server x86_64 13.16-1.el9 appstream 5.8 M 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs x86_64 2:19.3.0-5120.g18432218.el9 ceph 161 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados x86_64 2:19.3.0-5120.g18432218.el9 ceph 321 k 2024-09-21T02:47:18.538 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 307 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw x86_64 2:19.3.0-5120.g18432218.el9 ceph 99 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse x86_64 2:19.3.0-5120.g18432218.el9 ceph 86 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.1 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd x86_64 2:19.3.0-5120.g18432218.el9 ceph 170 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel x86_64 6.4.0-4.el9 appstream 96 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrading: 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager x86_64 1:1.51.0-1.el9 baseos 2.3 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm x86_64 1:1.51.0-1.el9 baseos 1.8 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team x86_64 1:1.51.0-1.el9 baseos 41 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui x86_64 1:1.51.0-1.el9 baseos 249 k 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: librados2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.4 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 3.2 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: openssl x86_64 1:3.2.2-6.el9 baseos 1.4 M 2024-09-21T02:47:18.539 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs x86_64 1:3.2.2-6.el9 baseos 2.4 M 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout:Installing dependencies: 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 20 M 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 34 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds x86_64 2:19.3.0-5120.g18432218.el9 ceph 2.3 M 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 292 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.0 M 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd x86_64 2:19.3.0-5120.g18432218.el9 ceph 18 M 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts noarch 2:19.3.0-5120.g18432218.el9 ceph-noarch 16 k 2024-09-21T02:47:18.540 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux x86_64 2:19.3.0-5120.g18432218.el9 ceph 25 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: fuse x86_64 2.9.9-16.el9 baseos 80 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite x86_64 2:19.3.0-5120.g18432218.el9 ceph 158 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran x86_64 11.5.0-2.el9 baseos 801 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: libnbd x86_64 1.20.2-2.el9 appstream 171 k 2024-09-21T02:47:18.541 INFO:teuthology.orchestra.run.smithi018.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath x86_64 11.5.0-2.el9 baseos 191 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1 x86_64 2:19.3.0-5120.g18432218.el9 ceph 507 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2 x86_64 2:19.3.0-5120.g18432218.el9 ceph 5.3 M 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: openblas x86_64 0.3.26-2.el9 appstream 38 k 2024-09-21T02:47:18.542 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp x86_64 0.3.26-2.el9 appstream 4.9 M 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs x86_64 13.16-1.el9 appstream 137 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse x86_64 2:19.3.0-5120.g18432218.el9 ceph 45 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common x86_64 2:19.3.0-5120.g18432218.el9 ceph 148 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-09-21T02:47:18.543 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama noarch 0.4.6-3.el9 appstream 61 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel x86_64 3.9.19-8.el9 appstream 245 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth noarch 1:2.34.0-1.el9 epel 220 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-09-21T02:47:18.544 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2 noarch 2.11.3-6.el9 appstream 248 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-09-21T02:47:18.545 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-09-21T02:47:18.546 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions noarch 4.12.2-2.el9 epel 74 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3 noarch 1.26.5-6.el9 baseos 215 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob noarch 1.8.8-2.el9 epel 230 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-09-21T02:47:18.547 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: socat x86_64 1.7.4.1-6.el9 appstream 304 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout:Installing weak dependencies: 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction Summary 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout:======================================================================================= 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout:Install 149 Packages 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout:Upgrade 8 Packages 2024-09-21T02:47:18.548 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:47:18.549 INFO:teuthology.orchestra.run.smithi123.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 1.0 MB/s | 125 kB 00:00 2024-09-21T02:47:18.549 INFO:teuthology.orchestra.run.smithi018.stdout:Total download size: 228 M 2024-09-21T02:47:18.549 INFO:teuthology.orchestra.run.smithi018.stdout:Downloading Packages: 2024-09-21T02:47:18.561 INFO:teuthology.orchestra.run.smithi016.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 19 MB/s | 10 MB 00:00 2024-09-21T02:47:18.604 INFO:teuthology.orchestra.run.smithi123.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.7 MB/s | 215 kB 00:00 2024-09-21T02:47:18.753 INFO:teuthology.orchestra.run.smithi016.stdout:(11/157): ceph-osd-19.3.0-5120.g18432218.el9.x8 15 MB/s | 18 MB 00:01 2024-09-21T02:47:18.754 INFO:teuthology.orchestra.run.smithi123.stdout:(48/157): automake-1.16.2-8.el9.noarch.rpm 3.1 MB/s | 689 kB 00:00 2024-09-21T02:47:18.778 INFO:teuthology.orchestra.run.smithi016.stdout:(12/157): cephfs-mirror-19.3.0-5120.g18432218.e 1.0 MB/s | 226 kB 00:00 2024-09-21T02:47:18.804 INFO:teuthology.orchestra.run.smithi016.stdout:(13/157): cephfs-shell-19.3.0-5120.g18432218.el 539 kB/s | 27 kB 00:00 2024-09-21T02:47:18.804 INFO:teuthology.orchestra.run.smithi123.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 2.1 MB/s | 107 kB 00:00 2024-09-21T02:47:18.828 INFO:teuthology.orchestra.run.smithi016.stdout:(14/157): libcephfs-devel-19.3.0-5120.g18432218 632 kB/s | 32 kB 00:00 2024-09-21T02:47:18.846 INFO:teuthology.orchestra.run.smithi123.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 522 kB/s | 22 kB 00:00 2024-09-21T02:47:18.862 INFO:teuthology.orchestra.run.smithi016.stdout:(15/157): libcephfs2-19.3.0-5120.g18432218.el9. 13 MB/s | 766 kB 00:00 2024-09-21T02:47:18.880 INFO:teuthology.orchestra.run.smithi123.stdout:(51/157): bison-3.7.4-5.el9.x86_64.rpm 3.3 MB/s | 943 kB 00:00 2024-09-21T02:47:18.887 INFO:teuthology.orchestra.run.smithi016.stdout:(16/157): libcephsqlite-19.3.0-5120.g18432218.e 2.6 MB/s | 158 kB 00:00 2024-09-21T02:47:18.905 INFO:teuthology.orchestra.run.smithi123.stdout:(52/157): flex-2.6.4-9.el9.x86_64.rpm 5.2 MB/s | 313 kB 00:00 2024-09-21T02:47:18.921 INFO:teuthology.orchestra.run.smithi016.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 2.1 MB/s | 127 kB 00:00 2024-09-21T02:47:18.928 INFO:teuthology.orchestra.run.smithi018.stdout:(1/157): ceph-19.3.0-5120.g18432218.el9.x86_64. 55 kB/s | 6.6 kB 00:00 2024-09-21T02:47:18.931 INFO:teuthology.orchestra.run.smithi123.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-09-21T02:47:18.955 INFO:teuthology.orchestra.run.smithi016.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 7.4 MB/s | 507 kB 00:00 2024-09-21T02:47:18.972 INFO:teuthology.orchestra.run.smithi123.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 438 kB/s | 18 kB 00:00 2024-09-21T02:47:19.015 INFO:teuthology.orchestra.run.smithi123.stdout:(55/157): libaio-devel-0.3.111-13.el9.x86_64.rp 268 kB/s | 11 kB 00:00 2024-09-21T02:47:19.022 INFO:teuthology.orchestra.run.smithi016.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 678 kB/s | 45 kB 00:00 2024-09-21T02:47:19.065 INFO:teuthology.orchestra.run.smithi123.stdout:(56/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-09-21T02:47:19.090 INFO:teuthology.orchestra.run.smithi123.stdout:(57/157): libnbd-1.20.2-2.el9.x86_64.rpm 2.2 MB/s | 171 kB 00:00 2024-09-21T02:47:19.123 INFO:teuthology.orchestra.run.smithi123.stdout:(58/157): python3-cryptography-36.0.1-4.el9.x86 1.4 MB/s | 1.2 MB 00:00 2024-09-21T02:47:19.149 INFO:teuthology.orchestra.run.smithi123.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 775 kB/s | 45 kB 00:00 2024-09-21T02:47:19.174 INFO:teuthology.orchestra.run.smithi123.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2024-09-21T02:47:19.199 INFO:teuthology.orchestra.run.smithi123.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 11 MB/s | 581 kB 00:00 2024-09-21T02:47:19.224 INFO:teuthology.orchestra.run.smithi123.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 489 kB/s | 24 kB 00:00 2024-09-21T02:47:19.242 INFO:teuthology.orchestra.run.smithi123.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 1.1 MB/s | 47 kB 00:00 2024-09-21T02:47:19.275 INFO:teuthology.orchestra.run.smithi123.stdout:(64/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 5.7 MB/s | 292 kB 00:00 2024-09-21T02:47:19.300 INFO:teuthology.orchestra.run.smithi123.stdout:(65/157): lua-5.4.4-4.el9.x86_64.rpm 3.1 MB/s | 188 kB 00:00 2024-09-21T02:47:19.314 INFO:teuthology.orchestra.run.smithi016.stdout:(20/157): librgw2-19.3.0-5120.g18432218.el9.x86 13 MB/s | 5.3 MB 00:00 2024-09-21T02:47:19.317 INFO:teuthology.orchestra.run.smithi123.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 918 kB/s | 38 kB 00:00 2024-09-21T02:47:19.320 INFO:teuthology.orchestra.run.smithi018.stdout:(2/157): ceph-fuse-19.3.0-5120.g18432218.el9.x8 2.2 MB/s | 873 kB 00:00 2024-09-21T02:47:19.339 INFO:teuthology.orchestra.run.smithi016.stdout:(21/157): python3-ceph-common-19.3.0-5120.g1843 468 kB/s | 148 kB 00:00 2024-09-21T02:47:19.351 INFO:teuthology.orchestra.run.smithi123.stdout:(67/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.8 MB/s | 662 kB 00:00 2024-09-21T02:47:19.365 INFO:teuthology.orchestra.run.smithi016.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 3.1 MB/s | 161 kB 00:00 2024-09-21T02:47:19.387 INFO:teuthology.orchestra.run.smithi018.stdout:(3/157): ceph-immutable-object-cache-19.3.0-512 2.2 MB/s | 148 kB 00:00 2024-09-21T02:47:19.390 INFO:teuthology.orchestra.run.smithi016.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.2 MB/s | 321 kB 00:00 2024-09-21T02:47:19.415 INFO:teuthology.orchestra.run.smithi016.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 6.0 MB/s | 307 kB 00:00 2024-09-21T02:47:19.441 INFO:teuthology.orchestra.run.smithi016.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.9 MB/s | 99 kB 00:00 2024-09-21T02:47:19.451 INFO:teuthology.orchestra.run.smithi123.stdout:(68/157): openblas-openmp-0.3.26-2.el9.x86_64.r 33 MB/s | 4.9 MB 00:00 2024-09-21T02:47:19.466 INFO:teuthology.orchestra.run.smithi016.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T02:47:19.493 INFO:teuthology.orchestra.run.smithi123.stdout:(69/157): postgresql-13.16-1.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2024-09-21T02:47:19.533 INFO:teuthology.orchestra.run.smithi016.stdout:(27/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 2.5 MB/s | 170 kB 00:00 2024-09-21T02:47:19.535 INFO:teuthology.orchestra.run.smithi123.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 11 MB/s | 910 kB 00:00 2024-09-21T02:47:19.560 INFO:teuthology.orchestra.run.smithi123.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.0 MB/s | 137 kB 00:00 2024-09-21T02:47:19.617 INFO:teuthology.orchestra.run.smithi016.stdout:(28/157): rbd-mirror-19.3.0-5120.g18432218.el9. 18 MB/s | 3.1 MB 00:00 2024-09-21T02:47:19.619 INFO:teuthology.orchestra.run.smithi123.stdout:(72/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 15 MB/s | 4.4 MB 00:00 2024-09-21T02:47:19.642 INFO:teuthology.orchestra.run.smithi016.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 315 kB/s | 34 kB 00:00 2024-09-21T02:47:19.653 INFO:teuthology.orchestra.run.smithi123.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-09-21T02:47:19.676 INFO:teuthology.orchestra.run.smithi016.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 2.6 MB/s | 154 kB 00:00 2024-09-21T02:47:19.678 INFO:teuthology.orchestra.run.smithi123.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-09-21T02:47:19.720 INFO:teuthology.orchestra.run.smithi123.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.4 MB/s | 61 kB 00:00 2024-09-21T02:47:19.795 INFO:teuthology.orchestra.run.smithi123.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 22 MB/s | 5.8 MB 00:00 2024-09-21T02:47:19.821 INFO:teuthology.orchestra.run.smithi123.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.4 MB/s | 245 kB 00:00 2024-09-21T02:47:19.846 INFO:teuthology.orchestra.run.smithi123.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.8 MB/s | 248 kB 00:00 2024-09-21T02:47:19.871 INFO:teuthology.orchestra.run.smithi123.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-09-21T02:47:19.896 INFO:teuthology.orchestra.run.smithi123.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 382 kB/s | 19 kB 00:00 2024-09-21T02:47:19.922 INFO:teuthology.orchestra.run.smithi123.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 3.4 MB/s | 172 kB 00:00 2024-09-21T02:47:19.947 INFO:teuthology.orchestra.run.smithi123.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 693 kB/s | 35 kB 00:00 2024-09-21T02:47:20.039 INFO:teuthology.orchestra.run.smithi123.stdout:(83/157): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-09-21T02:47:20.070 INFO:teuthology.orchestra.run.smithi018.stdout:(4/157): ceph-base-19.3.0-5120.g18432218.el9.x8 4.4 MB/s | 5.5 MB 00:01 2024-09-21T02:47:20.072 INFO:teuthology.orchestra.run.smithi123.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.0 MB/s | 250 kB 00:00 2024-09-21T02:47:20.098 INFO:teuthology.orchestra.run.smithi123.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T02:47:20.123 INFO:teuthology.orchestra.run.smithi123.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.5 MB/s | 77 kB 00:00 2024-09-21T02:47:20.148 INFO:teuthology.orchestra.run.smithi123.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 843 kB/s | 42 kB 00:00 2024-09-21T02:47:20.154 INFO:teuthology.orchestra.run.smithi018.stdout:(5/157): ceph-mds-19.3.0-5120.g18432218.el9.x86 3.0 MB/s | 2.3 MB 00:00 2024-09-21T02:47:20.174 INFO:teuthology.orchestra.run.smithi123.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-09-21T02:47:20.199 INFO:teuthology.orchestra.run.smithi123.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-09-21T02:47:20.224 INFO:teuthology.orchestra.run.smithi123.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 5.4 MB/s | 279 kB 00:00 2024-09-21T02:47:20.242 INFO:teuthology.orchestra.run.smithi016.stdout:(31/157): ceph-mgr-diskprediction-local-19.3.0- 13 MB/s | 7.4 MB 00:00 2024-09-21T02:47:20.249 INFO:teuthology.orchestra.run.smithi123.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 1.0 MB/s | 51 kB 00:00 2024-09-21T02:47:20.279 INFO:teuthology.orchestra.run.smithi123.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 976 kB/s | 54 kB 00:00 2024-09-21T02:47:20.334 INFO:teuthology.orchestra.run.smithi016.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 3.1 MB/s | 292 kB 00:00 2024-09-21T02:47:20.346 INFO:teuthology.orchestra.run.smithi018.stdout:(6/157): ceph-mgr-19.3.0-5120.g18432218.el9.x86 5.5 MB/s | 1.5 MB 00:00 2024-09-21T02:47:20.350 INFO:teuthology.orchestra.run.smithi123.stdout:(93/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-09-21T02:47:20.375 INFO:teuthology.orchestra.run.smithi123.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 437 kB/s | 42 kB 00:00 2024-09-21T02:47:20.401 INFO:teuthology.orchestra.run.smithi123.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 882 kB/s | 44 kB 00:00 2024-09-21T02:47:20.401 INFO:teuthology.orchestra.run.smithi016.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 742 kB/s | 49 kB 00:00 2024-09-21T02:47:20.425 INFO:teuthology.orchestra.run.smithi123.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-09-21T02:47:20.451 INFO:teuthology.orchestra.run.smithi123.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 719 kB/s | 36 kB 00:00 2024-09-21T02:47:20.476 INFO:teuthology.orchestra.run.smithi123.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 946 kB/s | 47 kB 00:00 2024-09-21T02:47:20.501 INFO:teuthology.orchestra.run.smithi123.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.9 MB/s | 304 kB 00:00 2024-09-21T02:47:20.527 INFO:teuthology.orchestra.run.smithi123.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-09-21T02:47:20.552 INFO:teuthology.orchestra.run.smithi123.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.9 MB/s | 96 kB 00:00 2024-09-21T02:47:20.577 INFO:teuthology.orchestra.run.smithi016.stdout:(34/157): ceph-mgr-dashboard-19.3.0-5120.g18432 15 MB/s | 14 MB 00:00 2024-09-21T02:47:20.577 INFO:teuthology.orchestra.run.smithi123.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-09-21T02:47:20.594 INFO:teuthology.orchestra.run.smithi016.stdout:(35/157): ceph-prometheus-alerts-19.3.0-5120.g1 86 kB/s | 16 kB 00:00 2024-09-21T02:47:20.603 INFO:teuthology.orchestra.run.smithi123.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 892 kB/s | 45 kB 00:00 2024-09-21T02:47:20.711 INFO:teuthology.orchestra.run.smithi123.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 167 kB/s | 22 kB 00:00 2024-09-21T02:47:20.746 INFO:teuthology.orchestra.run.smithi018.stdout:(7/157): ceph-mon-19.3.0-5120.g18432218.el9.x86 8.4 MB/s | 5.0 MB 00:00 2024-09-21T02:47:20.887 INFO:teuthology.orchestra.run.smithi123.stdout:(105/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 26 MB/s | 16 MB 00:00 2024-09-21T02:47:20.937 INFO:teuthology.orchestra.run.smithi123.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 2.4 MB/s | 548 kB 00:00 2024-09-21T02:47:20.953 INFO:teuthology.orchestra.run.smithi123.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.3 MB/s | 19 kB 00:00 2024-09-21T02:47:20.961 INFO:teuthology.orchestra.run.smithi016.stdout:(36/157): ceph-test-19.3.0-5120.g18432218.el9.x 14 MB/s | 40 MB 00:02 2024-09-21T02:47:20.986 INFO:teuthology.orchestra.run.smithi016.stdout:(37/157): ceph-volume-19.3.0-5120.g18432218.el9 697 kB/s | 285 kB 00:00 2024-09-21T02:47:21.012 INFO:teuthology.orchestra.run.smithi123.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.4 MB/s | 308 kB 00:00 2024-09-21T02:47:21.019 INFO:teuthology.orchestra.run.smithi016.stdout:(38/157): cephadm-19.3.0-5120.g18432218.el9.noa 1.8 MB/s | 784 kB 00:00 2024-09-21T02:47:21.036 INFO:teuthology.orchestra.run.smithi016.stdout:(39/157): cephfs-top-19.3.0-5120.g18432218.el9. 244 kB/s | 18 kB 00:00 2024-09-21T02:47:21.046 INFO:teuthology.orchestra.run.smithi123.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 1.9 MB/s | 863 kB 00:00 2024-09-21T02:47:21.072 INFO:teuthology.orchestra.run.smithi123.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 442 kB/s | 26 kB 00:00 2024-09-21T02:47:21.105 INFO:teuthology.orchestra.run.smithi123.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2024-09-21T02:47:21.112 INFO:teuthology.orchestra.run.smithi016.stdout:(40/157): fuse-2.9.9-16.el9.x86_64.rpm 636 kB/s | 80 kB 00:00 2024-09-21T02:47:21.138 INFO:teuthology.orchestra.run.smithi123.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.7 MB/s | 151 kB 00:00 2024-09-21T02:47:21.164 INFO:teuthology.orchestra.run.smithi123.stdout:(113/157): liboath-2.6.7-2.el9.x86_64.rpm 414 kB/s | 49 kB 00:00 2024-09-21T02:47:21.196 INFO:teuthology.orchestra.run.smithi016.stdout:(41/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.5 MB/s | 801 kB 00:00 2024-09-21T02:47:21.205 INFO:teuthology.orchestra.run.smithi018.stdout:(8/157): ceph-osd-19.3.0-5120.g18432218.el9.x86 21 MB/s | 18 MB 00:00 2024-09-21T02:47:21.221 INFO:teuthology.orchestra.run.smithi016.stdout:(42/157): libquadmath-11.5.0-2.el9.x86_64.rpm 1.0 MB/s | 191 kB 00:00 2024-09-21T02:47:21.222 INFO:teuthology.orchestra.run.smithi123.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 9.8 MB/s | 839 kB 00:00 2024-09-21T02:47:21.246 INFO:teuthology.orchestra.run.smithi016.stdout:(43/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.8 MB/s | 253 kB 00:00 2024-09-21T02:47:21.255 INFO:teuthology.orchestra.run.smithi018.stdout:(9/157): ceph-selinux-19.3.0-5120.g18432218.el9 506 kB/s | 25 kB 00:00 2024-09-21T02:47:21.256 INFO:teuthology.orchestra.run.smithi123.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.3 MB/s | 43 kB 00:00 2024-09-21T02:47:21.271 INFO:teuthology.orchestra.run.smithi016.stdout:(44/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T02:47:21.281 INFO:teuthology.orchestra.run.smithi123.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T02:47:21.297 INFO:teuthology.orchestra.run.smithi016.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-09-21T02:47:21.307 INFO:teuthology.orchestra.run.smithi123.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 563 kB/s | 14 kB 00:00 2024-09-21T02:47:21.322 INFO:teuthology.orchestra.run.smithi016.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T02:47:21.332 INFO:teuthology.orchestra.run.smithi123.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-09-21T02:47:21.347 INFO:teuthology.orchestra.run.smithi016.stdout:(47/157): python3-urllib3-1.26.5-6.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-09-21T02:47:21.389 INFO:teuthology.orchestra.run.smithi016.stdout:(48/157): python3-cryptography-36.0.1-4.el9.x86 6.3 MB/s | 1.2 MB 00:00 2024-09-21T02:47:21.407 INFO:teuthology.orchestra.run.smithi123.stdout:(119/157): libarrow-9.0.0-11.el9.x86_64.rpm 9.7 MB/s | 4.4 MB 00:00 2024-09-21T02:47:21.432 INFO:teuthology.orchestra.run.smithi123.stdout:(120/157): python3-cheroot-8.6.0-4.el9.noarch.r 1.3 MB/s | 172 kB 00:00 2024-09-21T02:47:21.457 INFO:teuthology.orchestra.run.smithi123.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 2.8 MB/s | 358 kB 00:00 2024-09-21T02:47:21.480 INFO:teuthology.orchestra.run.smithi018.stdout:(10/157): ceph-radosgw-19.3.0-5120.g18432218.el 14 MB/s | 10 MB 00:00 2024-09-21T02:47:21.483 INFO:teuthology.orchestra.run.smithi123.stdout:(122/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-09-21T02:47:21.508 INFO:teuthology.orchestra.run.smithi123.stdout:(123/157): python3-google-auth-2.34.0-1.el9.noa 2.8 MB/s | 220 kB 00:00 2024-09-21T02:47:21.530 INFO:teuthology.orchestra.run.smithi018.stdout:(11/157): cephfs-mirror-19.3.0-5120.g18432218.e 4.4 MB/s | 226 kB 00:00 2024-09-21T02:47:21.533 INFO:teuthology.orchestra.run.smithi123.stdout:(124/157): python3-grpcio-tools-1.46.7-10.el9.x 2.8 MB/s | 144 kB 00:00 2024-09-21T02:47:21.556 INFO:teuthology.orchestra.run.smithi016.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 641 kB/s | 107 kB 00:00 2024-09-21T02:47:21.559 INFO:teuthology.orchestra.run.smithi123.stdout:(125/157): python3-jaraco-8.2.1-3.el9.noarch.rp 213 kB/s | 11 kB 00:00 2024-09-21T02:47:21.584 INFO:teuthology.orchestra.run.smithi123.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 715 kB/s | 18 kB 00:00 2024-09-21T02:47:21.598 INFO:teuthology.orchestra.run.smithi018.stdout:(12/157): cephfs-shell-19.3.0-5120.g18432218.el 405 kB/s | 27 kB 00:00 2024-09-21T02:47:21.609 INFO:teuthology.orchestra.run.smithi123.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 464 kB/s | 23 kB 00:00 2024-09-21T02:47:21.615 INFO:teuthology.orchestra.run.smithi016.stdout:(50/157): automake-1.16.2-8.el9.noarch.rpm 2.3 MB/s | 689 kB 00:00 2024-09-21T02:47:21.634 INFO:teuthology.orchestra.run.smithi123.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 387 kB/s | 19 kB 00:00 2024-09-21T02:47:21.640 INFO:teuthology.orchestra.run.smithi016.stdout:(51/157): elfutils-libelf-devel-0.191-4.el9.x86 260 kB/s | 22 kB 00:00 2024-09-21T02:47:21.660 INFO:teuthology.orchestra.run.smithi123.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 391 kB/s | 20 kB 00:00 2024-09-21T02:47:21.664 INFO:teuthology.orchestra.run.smithi018.stdout:(13/157): libcephfs-devel-19.3.0-5120.g18432218 478 kB/s | 32 kB 00:00 2024-09-21T02:47:21.674 INFO:teuthology.orchestra.run.smithi016.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-09-21T02:47:21.693 INFO:teuthology.orchestra.run.smithi123.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.4 MB/s | 46 kB 00:00 2024-09-21T02:47:21.699 INFO:teuthology.orchestra.run.smithi016.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 554 kB/s | 33 kB 00:00 2024-09-21T02:47:21.718 INFO:teuthology.orchestra.run.smithi123.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 3.1 MB/s | 79 kB 00:00 2024-09-21T02:47:21.725 INFO:teuthology.orchestra.run.smithi016.stdout:(54/157): flex-2.6.4-9.el9.x86_64.rpm 2.8 MB/s | 313 kB 00:00 2024-09-21T02:47:21.744 INFO:teuthology.orchestra.run.smithi123.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:47:21.748 INFO:teuthology.orchestra.run.smithi018.stdout:(14/157): libcephfs2-19.3.0-5120.g18432218.el9. 9.0 MB/s | 766 kB 00:00 2024-09-21T02:47:21.750 INFO:teuthology.orchestra.run.smithi016.stdout:(55/157): flexiblas-openblas-openmp-3.0.4-8.el9 363 kB/s | 18 kB 00:00 2024-09-21T02:47:21.775 INFO:teuthology.orchestra.run.smithi016.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 222 kB/s | 11 kB 00:00 2024-09-21T02:47:21.777 INFO:teuthology.orchestra.run.smithi123.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 7.2 MB/s | 1.0 MB 00:00 2024-09-21T02:47:21.803 INFO:teuthology.orchestra.run.smithi123.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 659 kB/s | 16 kB 00:00 2024-09-21T02:47:21.823 INFO:teuthology.orchestra.run.smithi018.stdout:(15/157): libcephsqlite-19.3.0-5120.g18432218.e 2.1 MB/s | 158 kB 00:00 2024-09-21T02:47:21.828 INFO:teuthology.orchestra.run.smithi123.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 3.2 MB/s | 272 kB 00:00 2024-09-21T02:47:21.834 INFO:teuthology.orchestra.run.smithi016.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 19 MB/s | 3.0 MB 00:00 2024-09-21T02:47:21.853 INFO:teuthology.orchestra.run.smithi123.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.7 MB/s | 90 kB 00:00 2024-09-21T02:47:21.867 INFO:teuthology.orchestra.run.smithi016.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 1.4 MB/s | 171 kB 00:00 2024-09-21T02:47:21.878 INFO:teuthology.orchestra.run.smithi123.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-21T02:47:21.893 INFO:teuthology.orchestra.run.smithi016.stdout:(59/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 772 kB/s | 45 kB 00:00 2024-09-21T02:47:21.918 INFO:teuthology.orchestra.run.smithi016.stdout:(60/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2024-09-21T02:47:21.929 INFO:teuthology.orchestra.run.smithi123.stdout:(138/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.3 MB/s | 2.0 MB 00:00 2024-09-21T02:47:21.954 INFO:teuthology.orchestra.run.smithi123.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-09-21T02:47:21.960 INFO:teuthology.orchestra.run.smithi016.stdout:(61/157): libtool-2.4.6-46.el9.x86_64.rpm 8.5 MB/s | 581 kB 00:00 2024-09-21T02:47:21.979 INFO:teuthology.orchestra.run.smithi123.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.1 MB/s | 59 kB 00:00 2024-09-21T02:47:21.985 INFO:teuthology.orchestra.run.smithi016.stdout:(62/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 366 kB/s | 24 kB 00:00 2024-09-21T02:47:22.005 INFO:teuthology.orchestra.run.smithi123.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 712 kB/s | 36 kB 00:00 2024-09-21T02:47:22.010 INFO:teuthology.orchestra.run.smithi016.stdout:(63/157): librdkafka-1.6.1-102.el9.x86_64.rpm 4.5 MB/s | 662 kB 00:00 2024-09-21T02:47:22.015 INFO:teuthology.orchestra.run.smithi018.stdout:(16/157): ceph-common-19.3.0-5120.g18432218.el9 6.2 MB/s | 20 MB 00:03 2024-09-21T02:47:22.030 INFO:teuthology.orchestra.run.smithi123.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.2 MB/s | 188 kB 00:00 2024-09-21T02:47:22.036 INFO:teuthology.orchestra.run.smithi016.stdout:(64/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 626 kB/s | 47 kB 00:00 2024-09-21T02:47:22.040 INFO:teuthology.orchestra.run.smithi018.stdout:(17/157): librados-devel-19.3.0-5120.g18432218. 584 kB/s | 127 kB 00:00 2024-09-21T02:47:22.055 INFO:teuthology.orchestra.run.smithi123.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 987 kB/s | 74 kB 00:00 2024-09-21T02:47:22.078 INFO:teuthology.orchestra.run.smithi016.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2024-09-21T02:47:22.080 INFO:teuthology.orchestra.run.smithi123.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.7 MB/s | 90 kB 00:00 2024-09-21T02:47:22.103 INFO:teuthology.orchestra.run.smithi016.stdout:(66/157): openblas-0.3.26-2.el9.x86_64.rpm 577 kB/s | 38 kB 00:00 2024-09-21T02:47:22.114 INFO:teuthology.orchestra.run.smithi123.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-09-21T02:47:22.124 INFO:teuthology.orchestra.run.smithi018.stdout:(18/157): libradosstriper1-19.3.0-5120.g1843221 4.6 MB/s | 507 kB 00:00 2024-09-21T02:47:22.128 INFO:teuthology.orchestra.run.smithi016.stdout:(67/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T02:47:22.139 INFO:teuthology.orchestra.run.smithi123.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 341 kB/s | 20 kB 00:00 2024-09-21T02:47:22.165 INFO:teuthology.orchestra.run.smithi123.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T02:47:22.190 INFO:teuthology.orchestra.run.smithi123.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.1 MB/s | 427 kB 00:00 2024-09-21T02:47:22.216 INFO:teuthology.orchestra.run.smithi018.stdout:(19/157): python3-ceph-argparse-19.3.0-5120.g18 492 kB/s | 45 kB 00:00 2024-09-21T02:47:22.237 INFO:teuthology.orchestra.run.smithi016.stdout:(68/157): postgresql-13.16-1.el9.x86_64.rpm 15 MB/s | 1.6 MB 00:00 2024-09-21T02:47:22.265 INFO:teuthology.orchestra.run.smithi123.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 13 MB/s | 1.6 MB 00:00 2024-09-21T02:47:22.316 INFO:teuthology.orchestra.run.smithi018.stdout:(20/157): python3-ceph-common-19.3.0-5120.g1843 1.5 MB/s | 148 kB 00:00 2024-09-21T02:47:22.354 INFO:teuthology.orchestra.run.smithi016.stdout:(69/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-09-21T02:47:22.387 INFO:teuthology.orchestra.run.smithi016.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 5.9 MB/s | 910 kB 00:00 2024-09-21T02:47:22.413 INFO:teuthology.orchestra.run.smithi016.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 2.3 MB/s | 137 kB 00:00 2024-09-21T02:47:22.416 INFO:teuthology.orchestra.run.smithi018.stdout:(21/157): librgw2-19.3.0-5120.g18432218.el9.x86 14 MB/s | 5.3 MB 00:00 2024-09-21T02:47:22.442 INFO:teuthology.orchestra.run.smithi018.stdout:(22/157): python3-cephfs-19.3.0-5120.g18432218. 1.3 MB/s | 161 kB 00:00 2024-09-21T02:47:22.467 INFO:teuthology.orchestra.run.smithi018.stdout:(23/157): python3-rados-19.3.0-5120.g18432218.e 6.3 MB/s | 321 kB 00:00 2024-09-21T02:47:22.480 INFO:teuthology.orchestra.run.smithi016.stdout:(72/157): openblas-openmp-0.3.26-2.el9.x86_64.r 12 MB/s | 4.9 MB 00:00 2024-09-21T02:47:22.501 INFO:teuthology.orchestra.run.smithi018.stdout:(24/157): python3-rbd-19.3.0-5120.g18432218.el9 5.1 MB/s | 307 kB 00:00 2024-09-21T02:47:22.522 INFO:teuthology.orchestra.run.smithi016.stdout:(73/157): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-09-21T02:47:22.526 INFO:teuthology.orchestra.run.smithi018.stdout:(25/157): python3-rgw-19.3.0-5120.g18432218.el9 1.7 MB/s | 99 kB 00:00 2024-09-21T02:47:22.547 INFO:teuthology.orchestra.run.smithi016.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 1.3 MB/s | 87 kB 00:00 2024-09-21T02:47:22.552 INFO:teuthology.orchestra.run.smithi018.stdout:(26/157): rbd-fuse-19.3.0-5120.g18432218.el9.x8 1.7 MB/s | 86 kB 00:00 2024-09-21T02:47:22.574 INFO:teuthology.orchestra.run.smithi123.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 8.3 MB/s | 3.4 MB 00:00 2024-09-21T02:47:22.597 INFO:teuthology.orchestra.run.smithi016.stdout:(75/157): python3-colorama-0.4.6-3.el9.noarch.r 1.2 MB/s | 61 kB 00:00 2024-09-21T02:47:22.706 INFO:teuthology.orchestra.run.smithi016.stdout:(76/157): postgresql-server-13.16-1.el9.x86_64. 18 MB/s | 5.8 MB 00:00 2024-09-21T02:47:22.731 INFO:teuthology.orchestra.run.smithi016.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 1.8 MB/s | 245 kB 00:00 2024-09-21T02:47:22.765 INFO:teuthology.orchestra.run.smithi016.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 4.1 MB/s | 248 kB 00:00 2024-09-21T02:47:22.782 INFO:teuthology.orchestra.run.smithi016.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 509 kB/s | 26 kB 00:00 2024-09-21T02:47:22.865 INFO:teuthology.orchestra.run.smithi016.stdout:(80/157): python3-babel-2.9.1-2.el9.noarch.rpm 17 MB/s | 6.0 MB 00:00 2024-09-21T02:47:22.866 INFO:teuthology.orchestra.run.smithi123.stdout:(151/157): librbd1-19.3.0-5120.g18432218.el9.x8 4.7 MB/s | 3.2 MB 00:00 2024-09-21T02:47:22.899 INFO:teuthology.orchestra.run.smithi016.stdout:(81/157): python3-jsonpointer-2.0-4.el9.noarch. 143 kB/s | 19 kB 00:00 2024-09-21T02:47:22.910 INFO:teuthology.orchestra.run.smithi018.stdout:(27/157): ceph-test-19.3.0-5120.g18432218.el9.x 24 MB/s | 40 MB 00:01 2024-09-21T02:47:22.924 INFO:teuthology.orchestra.run.smithi016.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 590 kB/s | 35 kB 00:00 2024-09-21T02:47:22.935 INFO:teuthology.orchestra.run.smithi018.stdout:(28/157): rbd-nbd-19.3.0-5120.g18432218.el9.x86 444 kB/s | 170 kB 00:00 2024-09-21T02:47:22.950 INFO:teuthology.orchestra.run.smithi016.stdout:(83/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-09-21T02:47:22.952 INFO:teuthology.orchestra.run.smithi018.stdout:(29/157): ceph-grafana-dashboards-19.3.0-5120.g 823 kB/s | 34 kB 00:00 2024-09-21T02:47:22.983 INFO:teuthology.orchestra.run.smithi016.stdout:(84/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 4.2 MB/s | 250 kB 00:00 2024-09-21T02:47:22.986 INFO:teuthology.orchestra.run.smithi018.stdout:(30/157): ceph-mgr-cephadm-19.3.0-5120.g1843221 3.0 MB/s | 154 kB 00:00 2024-09-21T02:47:23.009 INFO:teuthology.orchestra.run.smithi016.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-09-21T02:47:23.043 INFO:teuthology.orchestra.run.smithi016.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-09-21T02:47:23.053 INFO:teuthology.orchestra.run.smithi018.stdout:(31/157): rbd-mirror-19.3.0-5120.g18432218.el9. 5.9 MB/s | 3.1 MB 00:00 2024-09-21T02:47:23.068 INFO:teuthology.orchestra.run.smithi016.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 719 kB/s | 42 kB 00:00 2024-09-21T02:47:23.101 INFO:teuthology.orchestra.run.smithi016.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 4.5 MB/s | 269 kB 00:00 2024-09-21T02:47:23.103 INFO:teuthology.orchestra.run.smithi018.stdout:(32/157): ceph-mgr-modules-core-19.3.0-5120.g18 5.7 MB/s | 292 kB 00:00 2024-09-21T02:47:23.127 INFO:teuthology.orchestra.run.smithi016.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.6 MB/s | 159 kB 00:00 2024-09-21T02:47:23.170 INFO:teuthology.orchestra.run.smithi018.stdout:(33/157): ceph-mgr-rook-19.3.0-5120.g18432218.e 743 kB/s | 49 kB 00:00 2024-09-21T02:47:23.202 INFO:teuthology.orchestra.run.smithi016.stdout:(90/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 17 MB/s | 5.1 MB 00:00 2024-09-21T02:47:23.227 INFO:teuthology.orchestra.run.smithi016.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 508 kB/s | 51 kB 00:00 2024-09-21T02:47:23.237 INFO:teuthology.orchestra.run.smithi018.stdout:(34/157): ceph-prometheus-alerts-19.3.0-5120.g1 247 kB/s | 16 kB 00:00 2024-09-21T02:47:23.252 INFO:teuthology.orchestra.run.smithi016.stdout:(92/157): python3-pyasn1-modules-0.4.8-6.el9.no 1.8 MB/s | 279 kB 00:00 2024-09-21T02:47:23.258 INFO:teuthology.orchestra.run.smithi123.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 104 kB/s | 41 kB 00:00 2024-09-21T02:47:23.278 INFO:teuthology.orchestra.run.smithi016.stdout:(93/157): python3-requests-oauthlib-1.3.0-12.el 710 kB/s | 54 kB 00:00 2024-09-21T02:47:23.303 INFO:teuthology.orchestra.run.smithi016.stdout:(94/157): python3-toml-0.10.2-6.el9.noarch.rpm 827 kB/s | 42 kB 00:00 2024-09-21T02:47:23.337 INFO:teuthology.orchestra.run.smithi016.stdout:(95/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 755 kB/s | 44 kB 00:00 2024-09-21T02:47:23.362 INFO:teuthology.orchestra.run.smithi016.stdout:(96/157): qatlib-24.02.0-1.el9.x86_64.rpm 3.7 MB/s | 221 kB 00:00 2024-09-21T02:47:23.374 INFO:teuthology.orchestra.run.smithi123.stdout:(153/157): NetworkManager-1.51.0-1.el9.x86_64.r 2.1 MB/s | 2.3 MB 00:01 2024-09-21T02:47:23.387 INFO:teuthology.orchestra.run.smithi016.stdout:(97/157): qatlib-service-24.02.0-1.el9.x86_64.r 714 kB/s | 36 kB 00:00 2024-09-21T02:47:23.404 INFO:teuthology.orchestra.run.smithi018.stdout:(35/157): ceph-mgr-dashboard-19.3.0-5120.g18432 31 MB/s | 14 MB 00:00 2024-09-21T02:47:23.413 INFO:teuthology.orchestra.run.smithi016.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-09-21T02:47:23.429 INFO:teuthology.orchestra.run.smithi018.stdout:(36/157): ceph-volume-19.3.0-5120.g18432218.el9 1.5 MB/s | 285 kB 00:00 2024-09-21T02:47:23.446 INFO:teuthology.orchestra.run.smithi016.stdout:(99/157): socat-1.7.4.1-6.el9.x86_64.rpm 5.1 MB/s | 304 kB 00:00 2024-09-21T02:47:23.471 INFO:teuthology.orchestra.run.smithi018.stdout:(37/157): cephadm-19.3.0-5120.g18432218.el9.noa 12 MB/s | 784 kB 00:00 2024-09-21T02:47:23.472 INFO:teuthology.orchestra.run.smithi016.stdout:(100/157): uuid-1.6.2-55.el9.x86_64.rpm 985 kB/s | 58 kB 00:00 2024-09-21T02:47:23.496 INFO:teuthology.orchestra.run.smithi018.stdout:(38/157): cephfs-top-19.3.0-5120.g18432218.el9. 276 kB/s | 18 kB 00:00 2024-09-21T02:47:23.505 INFO:teuthology.orchestra.run.smithi016.stdout:(101/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 1.6 MB/s | 96 kB 00:00 2024-09-21T02:47:23.516 INFO:teuthology.orchestra.run.smithi123.stdout:(154/157): openssl-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 1.4 MB 00:00 2024-09-21T02:47:23.530 INFO:teuthology.orchestra.run.smithi016.stdout:(102/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-09-21T02:47:23.564 INFO:teuthology.orchestra.run.smithi016.stdout:(103/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 766 kB/s | 45 kB 00:00 2024-09-21T02:47:23.613 INFO:teuthology.orchestra.run.smithi018.stdout:(39/157): fuse-2.9.9-16.el9.x86_64.rpm 563 kB/s | 80 kB 00:00 2024-09-21T02:47:23.631 INFO:teuthology.orchestra.run.smithi016.stdout:(104/157): lua-devel-5.4.4-4.el9.x86_64.rpm 222 kB/s | 22 kB 00:00 2024-09-21T02:47:23.650 INFO:teuthology.orchestra.run.smithi123.stdout:(155/157): NetworkManager-tui-1.51.0-1.el9.x86_ 635 kB/s | 249 kB 00:00 2024-09-21T02:47:23.689 INFO:teuthology.orchestra.run.smithi018.stdout:(40/157): libgfortran-11.5.0-2.el9.x86_64.rpm 4.1 MB/s | 801 kB 00:00 2024-09-21T02:47:23.698 INFO:teuthology.orchestra.run.smithi016.stdout:(105/157): abseil-cpp-20211102.0-3.el9.x86_64.r 8.0 MB/s | 548 kB 00:00 2024-09-21T02:47:23.705 INFO:teuthology.orchestra.run.smithi018.stdout:(41/157): libquadmath-11.5.0-2.el9.x86_64.rpm 2.0 MB/s | 191 kB 00:00 2024-09-21T02:47:23.732 INFO:teuthology.orchestra.run.smithi016.stdout:(106/157): gperftools-libs-2.9.1-3.el9.x86_64.r 9.1 MB/s | 308 kB 00:00 2024-09-21T02:47:23.739 INFO:teuthology.orchestra.run.smithi018.stdout:(42/157): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-09-21T02:47:23.757 INFO:teuthology.orchestra.run.smithi016.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 781 kB/s | 19 kB 00:00 2024-09-21T02:47:23.789 INFO:teuthology.orchestra.run.smithi018.stdout:(43/157): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-09-21T02:47:23.840 INFO:teuthology.orchestra.run.smithi018.stdout:(44/157): python3-cryptography-36.0.1-4.el9.x86 9.2 MB/s | 1.2 MB 00:00 2024-09-21T02:47:23.865 INFO:teuthology.orchestra.run.smithi018.stdout:(45/157): python3-pycparser-2.20-6.el9.noarch.r 1.8 MB/s | 135 kB 00:00 2024-09-21T02:47:23.890 INFO:teuthology.orchestra.run.smithi018.stdout:(46/157): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-09-21T02:47:23.924 INFO:teuthology.orchestra.run.smithi016.stdout:(108/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-21T02:47:23.966 INFO:teuthology.orchestra.run.smithi016.stdout:(109/157): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-09-21T02:47:23.982 INFO:teuthology.orchestra.run.smithi018.stdout:(47/157): ceph-mgr-diskprediction-local-19.3.0- 7.4 MB/s | 7.4 MB 00:00 2024-09-21T02:47:24.000 INFO:teuthology.orchestra.run.smithi016.stdout:(110/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 346 kB/s | 26 kB 00:00 2024-09-21T02:47:24.008 INFO:teuthology.orchestra.run.smithi018.stdout:(48/157): python3-urllib3-1.26.5-6.el9.noarch.r 1.5 MB/s | 215 kB 00:00 2024-09-21T02:47:24.025 INFO:teuthology.orchestra.run.smithi016.stdout:(111/157): libunwind-1.6.2-1.el9.x86_64.rpm 2.9 MB/s | 67 kB 00:00 2024-09-21T02:47:24.051 INFO:teuthology.orchestra.run.smithi016.stdout:(112/157): liboath-2.6.7-2.el9.x86_64.rpm 594 kB/s | 49 kB 00:00 2024-09-21T02:47:24.075 INFO:teuthology.orchestra.run.smithi123.stdout:(156/157): NetworkManager-libnm-1.51.0-1.el9.x8 1.2 MB/s | 1.8 MB 00:01 2024-09-21T02:47:24.076 INFO:teuthology.orchestra.run.smithi016.stdout:(113/157): luarocks-3.9.2-5.el9.noarch.rpm 2.9 MB/s | 151 kB 00:00 2024-09-21T02:47:24.118 INFO:teuthology.orchestra.run.smithi016.stdout:(114/157): python3-asyncssh-2.13.2-5.el9.noarch 13 MB/s | 548 kB 00:00 2024-09-21T02:47:24.143 INFO:teuthology.orchestra.run.smithi016.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:47:24.168 INFO:teuthology.orchestra.run.smithi016.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T02:47:24.185 INFO:teuthology.orchestra.run.smithi016.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 855 kB/s | 14 kB 00:00 2024-09-21T02:47:24.227 INFO:teuthology.orchestra.run.smithi016.stdout:(118/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 4.6 MB/s | 839 kB 00:00 2024-09-21T02:47:24.253 INFO:teuthology.orchestra.run.smithi016.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.5 MB/s | 172 kB 00:00 2024-09-21T02:47:24.286 INFO:teuthology.orchestra.run.smithi016.stdout:(120/157): python3-cherrypy-18.6.1-2.el9.noarch 6.0 MB/s | 358 kB 00:00 2024-09-21T02:47:24.312 INFO:teuthology.orchestra.run.smithi016.stdout:(121/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 5.1 MB/s | 303 kB 00:00 2024-09-21T02:47:24.337 INFO:teuthology.orchestra.run.smithi016.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:47:24.363 INFO:teuthology.orchestra.run.smithi016.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T02:47:24.388 INFO:teuthology.orchestra.run.smithi016.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 433 kB/s | 11 kB 00:00 2024-09-21T02:47:24.446 INFO:teuthology.orchestra.run.smithi016.stdout:(125/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.4 MB/s | 4.4 MB 00:00 2024-09-21T02:47:24.466 INFO:teuthology.orchestra.run.smithi018.stdout:(49/157): boost-program-options-1.75.0-8.el9.x8 233 kB/s | 107 kB 00:00 2024-09-21T02:47:24.471 INFO:teuthology.orchestra.run.smithi016.stdout:(126/157): python3-jaraco-classes-3.2.1-5.el9.n 211 kB/s | 18 kB 00:00 2024-09-21T02:47:24.497 INFO:teuthology.orchestra.run.smithi016.stdout:(127/157): python3-jaraco-collections-3.0.0-8.e 461 kB/s | 23 kB 00:00 2024-09-21T02:47:24.522 INFO:teuthology.orchestra.run.smithi016.stdout:(128/157): python3-jaraco-functools-3.5.0-2.el9 388 kB/s | 19 kB 00:00 2024-09-21T02:47:24.547 INFO:teuthology.orchestra.run.smithi016.stdout:(129/157): python3-jaraco-text-3.2.0-6.el9.noar 393 kB/s | 20 kB 00:00 2024-09-21T02:47:24.572 INFO:teuthology.orchestra.run.smithi016.stdout:(130/157): python3-logutils-0.3.5-21.el9.noarch 1.8 MB/s | 46 kB 00:00 2024-09-21T02:47:24.577 INFO:teuthology.orchestra.run.smithi018.stdout:(50/157): elfutils-libelf-devel-0.191-4.el9.x86 196 kB/s | 22 kB 00:00 2024-09-21T02:47:24.606 INFO:teuthology.orchestra.run.smithi016.stdout:(131/157): python3-more-itertools-8.12.0-2.el9. 2.3 MB/s | 79 kB 00:00 2024-09-21T02:47:24.631 INFO:teuthology.orchestra.run.smithi016.stdout:(132/157): python3-natsort-7.1.1-5.el9.noarch.r 2.3 MB/s | 58 kB 00:00 2024-09-21T02:47:24.724 INFO:teuthology.orchestra.run.smithi016.stdout:(133/157): python3-kubernetes-26.1.0-2.el9.noar 5.1 MB/s | 1.0 MB 00:00 2024-09-21T02:47:24.725 INFO:teuthology.orchestra.run.smithi123.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 2.0 MB/s | 2.4 MB 00:01 2024-09-21T02:47:24.732 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:47:24.732 INFO:teuthology.orchestra.run.smithi123.stdout:Total 21 MB/s | 228 MB 00:10 2024-09-21T02:47:24.752 INFO:teuthology.orchestra.run.smithi018.stdout:(51/157): automake-1.16.2-8.el9.noarch.rpm 800 kB/s | 689 kB 00:00 2024-09-21T02:47:24.757 INFO:teuthology.orchestra.run.smithi016.stdout:(134/157): python3-portend-3.1.0-2.el9.noarch.r 492 kB/s | 16 kB 00:00 2024-09-21T02:47:24.791 INFO:teuthology.orchestra.run.smithi016.stdout:(135/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.7 MB/s | 272 kB 00:00 2024-09-21T02:47:24.802 INFO:teuthology.orchestra.run.smithi018.stdout:(52/157): bison-3.7.4-5.el9.x86_64.rpm 1.1 MB/s | 943 kB 00:00 2024-09-21T02:47:24.816 INFO:teuthology.orchestra.run.smithi016.stdout:(136/157): python3-pyperclip-1.8.2-2.el9.noarch 1.1 MB/s | 28 kB 00:00 2024-09-21T02:47:24.842 INFO:teuthology.orchestra.run.smithi016.stdout:(137/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.0 MB/s | 90 kB 00:00 2024-09-21T02:47:24.861 INFO:teuthology.orchestra.run.smithi018.stdout:(53/157): flexiblas-3.0.4-8.el9.x86_64.rpm 300 kB/s | 33 kB 00:00 2024-09-21T02:47:24.862 INFO:teuthology.orchestra.run.smithi016.stdout:(138/157): python3-repoze-lru-0.7-16.el9.noarch 683 kB/s | 31 kB 00:00 2024-09-21T02:47:24.879 INFO:teuthology.orchestra.run.smithi016.stdout:(139/157): python3-rsa-4.9-2.el9.noarch.rpm 3.5 MB/s | 59 kB 00:00 2024-09-21T02:47:24.896 INFO:teuthology.orchestra.run.smithi016.stdout:(140/157): python3-tempora-5.0.0-2.el9.noarch.r 2.1 MB/s | 36 kB 00:00 2024-09-21T02:47:24.946 INFO:teuthology.orchestra.run.smithi016.stdout:(141/157): python3-grpcio-1.46.7-10.el9.x86_64. 3.2 MB/s | 2.0 MB 00:00 2024-09-21T02:47:24.970 INFO:teuthology.orchestra.run.smithi018.stdout:(54/157): flexiblas-openblas-openmp-3.0.4-8.el9 168 kB/s | 18 kB 00:00 2024-09-21T02:47:24.980 INFO:teuthology.orchestra.run.smithi016.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.3 MB/s | 188 kB 00:00 2024-09-21T02:47:24.995 INFO:teuthology.orchestra.run.smithi018.stdout:(55/157): flex-2.6.4-9.el9.x86_64.rpm 750 kB/s | 313 kB 00:00 2024-09-21T02:47:25.005 INFO:teuthology.orchestra.run.smithi016.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 684 kB/s | 74 kB 00:00 2024-09-21T02:47:25.030 INFO:teuthology.orchestra.run.smithi016.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 1.8 MB/s | 90 kB 00:00 2024-09-21T02:47:25.056 INFO:teuthology.orchestra.run.smithi016.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-09-21T02:47:25.070 INFO:teuthology.orchestra.run.smithi018.stdout:(56/157): libaio-devel-0.3.111-13.el9.x86_64.rp 111 kB/s | 11 kB 00:00 2024-09-21T02:47:25.081 INFO:teuthology.orchestra.run.smithi016.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 397 kB/s | 20 kB 00:00 2024-09-21T02:47:25.106 INFO:teuthology.orchestra.run.smithi016.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 3.7 MB/s | 191 kB 00:00 2024-09-21T02:47:25.131 INFO:teuthology.orchestra.run.smithi016.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-09-21T02:47:25.145 INFO:teuthology.orchestra.run.smithi018.stdout:(57/157): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.7 MB/s | 3.0 MB 00:00 2024-09-21T02:47:25.171 INFO:teuthology.orchestra.run.smithi018.stdout:(58/157): libnbd-1.20.2-2.el9.x86_64.rpm 972 kB/s | 171 kB 00:00 2024-09-21T02:47:25.182 INFO:teuthology.orchestra.run.smithi016.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T02:47:25.196 INFO:teuthology.orchestra.run.smithi018.stdout:(59/157): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.2 MB/s | 160 kB 00:00 2024-09-21T02:47:25.254 INFO:teuthology.orchestra.run.smithi018.stdout:(60/157): librabbitmq-0.11.0-7.el9.x86_64.rpm 415 kB/s | 45 kB 00:00 2024-09-21T02:47:25.357 INFO:teuthology.orchestra.run.smithi016.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T02:47:25.363 INFO:teuthology.orchestra.run.smithi018.stdout:(61/157): libuuid-devel-2.37.4-20.el9.x86_64.rp 226 kB/s | 24 kB 00:00 2024-09-21T02:47:25.424 INFO:teuthology.orchestra.run.smithi016.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 9.9 MB/s | 2.3 MB 00:00 2024-09-21T02:47:25.466 INFO:teuthology.orchestra.run.smithi016.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 986 kB/s | 41 kB 00:00 2024-09-21T02:47:25.480 INFO:teuthology.orchestra.run.smithi018.stdout:(62/157): librdkafka-1.6.1-102.el9.x86_64.rpm 2.1 MB/s | 662 kB 00:00 2024-09-21T02:47:25.505 INFO:teuthology.orchestra.run.smithi018.stdout:(63/157): libzstd-devel-1.5.1-2.el9.x86_64.rpm 334 kB/s | 47 kB 00:00 2024-09-21T02:47:25.516 INFO:teuthology.orchestra.run.smithi016.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 4.9 MB/s | 249 kB 00:00 2024-09-21T02:47:25.530 INFO:teuthology.orchestra.run.smithi018.stdout:(64/157): libtool-2.4.6-46.el9.x86_64.rpm 1.7 MB/s | 581 kB 00:00 2024-09-21T02:47:25.575 INFO:teuthology.orchestra.run.smithi016.stdout:(154/157): librbd1-19.3.0-5120.g18432218.el9.x8 7.1 MB/s | 3.2 MB 00:00 2024-09-21T02:47:25.597 INFO:teuthology.orchestra.run.smithi018.stdout:(65/157): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.4 MB/s | 292 kB 00:00 2024-09-21T02:47:25.623 INFO:teuthology.orchestra.run.smithi018.stdout:(66/157): lua-5.4.4-4.el9.x86_64.rpm 1.6 MB/s | 188 kB 00:00 2024-09-21T02:47:25.625 INFO:teuthology.orchestra.run.smithi016.stdout:(155/157): NetworkManager-libnm-1.51.0-1.el9.x8 6.8 MB/s | 1.8 MB 00:00 2024-09-21T02:47:25.648 INFO:teuthology.orchestra.run.smithi018.stdout:(67/157): openblas-0.3.26-2.el9.x86_64.rpm 328 kB/s | 38 kB 00:00 2024-09-21T02:47:25.667 INFO:teuthology.orchestra.run.smithi016.stdout:(156/157): openssl-3.2.2-6.el9.x86_64.rpm 9.0 MB/s | 1.4 MB 00:00 2024-09-21T02:47:25.826 INFO:teuthology.orchestra.run.smithi016.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 9.6 MB/s | 2.4 MB 00:00 2024-09-21T02:47:25.832 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:47:25.832 INFO:teuthology.orchestra.run.smithi016.stdout:Total 24 MB/s | 228 MB 00:09 2024-09-21T02:47:25.957 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2024-09-21T02:47:25.990 INFO:teuthology.orchestra.run.smithi018.stdout:(68/157): openssl-devel-3.2.2-6.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2024-09-21T02:47:26.109 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2024-09-21T02:47:26.109 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2024-09-21T02:47:26.149 INFO:teuthology.orchestra.run.smithi018.stdout:(69/157): openblas-openmp-0.3.26-2.el9.x86_64.r 9.0 MB/s | 4.9 MB 00:00 2024-09-21T02:47:26.182 INFO:teuthology.orchestra.run.smithi018.stdout:(70/157): postgresql-contrib-13.16-1.el9.x86_64 4.6 MB/s | 910 kB 00:00 2024-09-21T02:47:26.266 INFO:teuthology.orchestra.run.smithi018.stdout:(71/157): postgresql-private-libs-13.16-1.el9.x 1.1 MB/s | 137 kB 00:00 2024-09-21T02:47:26.391 INFO:teuthology.orchestra.run.smithi018.stdout:(72/157): protobuf-3.14.0-13.el9.x86_64.rpm 8.1 MB/s | 1.0 MB 00:00 2024-09-21T02:47:26.501 INFO:teuthology.orchestra.run.smithi018.stdout:(73/157): postgresql-13.16-1.el9.x86_64.rpm 1.9 MB/s | 1.6 MB 00:00 2024-09-21T02:47:26.525 INFO:teuthology.orchestra.run.smithi018.stdout:(74/157): python3-attrs-20.3.0-7.el9.noarch.rpm 654 kB/s | 87 kB 00:00 2024-09-21T02:47:26.659 INFO:teuthology.orchestra.run.smithi018.stdout:(75/157): postgresql-server-13.16-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-09-21T02:47:26.684 INFO:teuthology.orchestra.run.smithi018.stdout:(76/157): python3-colorama-0.4.6-3.el9.noarch.r 386 kB/s | 61 kB 00:00 2024-09-21T02:47:26.776 INFO:teuthology.orchestra.run.smithi018.stdout:(77/157): python3-devel-3.9.19-8.el9.x86_64.rpm 2.0 MB/s | 245 kB 00:00 2024-09-21T02:47:26.801 INFO:teuthology.orchestra.run.smithi018.stdout:(78/157): python3-jinja2-2.11.3-6.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-09-21T02:47:26.885 INFO:teuthology.orchestra.run.smithi018.stdout:(79/157): python3-jsonpatch-1.21-16.el9.noarch. 236 kB/s | 26 kB 00:00 2024-09-21T02:47:26.910 INFO:teuthology.orchestra.run.smithi018.stdout:(80/157): python3-jsonpointer-2.0-4.el9.noarch. 176 kB/s | 19 kB 00:00 2024-09-21T02:47:27.002 INFO:teuthology.orchestra.run.smithi018.stdout:(81/157): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-09-21T02:47:27.027 INFO:teuthology.orchestra.run.smithi018.stdout:(82/157): python3-markupsafe-1.1.1-12.el9.x86_6 297 kB/s | 35 kB 00:00 2024-09-21T02:47:27.057 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction check 2024-09-21T02:47:27.144 INFO:teuthology.orchestra.run.smithi018.stdout:(83/157): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.1 MB/s | 250 kB 00:00 2024-09-21T02:47:27.225 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction check succeeded. 2024-09-21T02:47:27.226 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction test 2024-09-21T02:47:27.270 INFO:teuthology.orchestra.run.smithi018.stdout:(84/157): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-09-21T02:47:27.295 INFO:teuthology.orchestra.run.smithi018.stdout:(85/157): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-09-21T02:47:27.387 INFO:teuthology.orchestra.run.smithi018.stdout:(86/157): python3-packaging-20.9-5.el9.noarch.r 661 kB/s | 77 kB 00:00 2024-09-21T02:47:27.412 INFO:teuthology.orchestra.run.smithi018.stdout:(87/157): python3-prettytable-0.7.2-27.el9.noar 362 kB/s | 42 kB 00:00 2024-09-21T02:47:27.504 INFO:teuthology.orchestra.run.smithi018.stdout:(88/157): python3-protobuf-3.14.0-13.el9.noarch 2.3 MB/s | 269 kB 00:00 2024-09-21T02:47:27.530 INFO:teuthology.orchestra.run.smithi018.stdout:(89/157): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.3 MB/s | 159 kB 00:00 2024-09-21T02:47:27.621 INFO:teuthology.orchestra.run.smithi018.stdout:(90/157): python3-pyasn1-modules-0.4.8-6.el9.no 2.3 MB/s | 279 kB 00:00 2024-09-21T02:47:27.647 INFO:teuthology.orchestra.run.smithi018.stdout:(91/157): python3-pytz-2021.1-5.el9.noarch.rpm 436 kB/s | 51 kB 00:00 2024-09-21T02:47:27.738 INFO:teuthology.orchestra.run.smithi018.stdout:(92/157): python3-requests-oauthlib-1.3.0-12.el 459 kB/s | 54 kB 00:00 2024-09-21T02:47:27.847 INFO:teuthology.orchestra.run.smithi018.stdout:(93/157): python3-toml-0.10.2-6.el9.noarch.rpm 385 kB/s | 42 kB 00:00 2024-09-21T02:47:27.913 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2024-09-21T02:47:27.913 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2024-09-21T02:47:27.964 INFO:teuthology.orchestra.run.smithi018.stdout:(94/157): python3-wcwidth-0.2.5-8.el9.noarch.rp 378 kB/s | 44 kB 00:00 2024-09-21T02:47:28.081 INFO:teuthology.orchestra.run.smithi018.stdout:(95/157): qatlib-24.02.0-1.el9.x86_64.rpm 1.8 MB/s | 221 kB 00:00 2024-09-21T02:47:28.198 INFO:teuthology.orchestra.run.smithi018.stdout:(96/157): qatlib-service-24.02.0-1.el9.x86_64.r 308 kB/s | 36 kB 00:00 2024-09-21T02:47:28.365 INFO:teuthology.orchestra.run.smithi018.stdout:(97/157): python3-scipy-1.6.2-8.el9.x86_64.rpm 23 MB/s | 16 MB 00:00 2024-09-21T02:47:28.390 INFO:teuthology.orchestra.run.smithi018.stdout:(98/157): qatzip-libs-1.2.0-1.el9.x86_64.rpm 247 kB/s | 47 kB 00:00 2024-09-21T02:47:28.474 INFO:teuthology.orchestra.run.smithi018.stdout:(99/157): python3-babel-2.9.1-2.el9.noarch.rpm 3.0 MB/s | 6.0 MB 00:01 2024-09-21T02:47:28.499 INFO:teuthology.orchestra.run.smithi018.stdout:(100/157): socat-1.7.4.1-6.el9.x86_64.rpm 2.2 MB/s | 304 kB 00:00 2024-09-21T02:47:28.516 INFO:teuthology.orchestra.run.smithi018.stdout:(101/157): uuid-1.6.2-55.el9.x86_64.rpm 460 kB/s | 58 kB 00:00 2024-09-21T02:47:28.583 INFO:teuthology.orchestra.run.smithi018.stdout:(102/157): xfsprogs-devel-6.4.0-4.el9.x86_64.rp 881 kB/s | 96 kB 00:00 2024-09-21T02:47:28.600 INFO:teuthology.orchestra.run.smithi018.stdout:(103/157): xmlstarlet-1.6.1-20.el9.x86_64.rpm 633 kB/s | 64 kB 00:00 2024-09-21T02:47:28.617 INFO:teuthology.orchestra.run.smithi018.stdout:(104/157): zlib-devel-1.2.11-41.el9.x86_64.rpm 445 kB/s | 45 kB 00:00 2024-09-21T02:47:28.642 INFO:teuthology.orchestra.run.smithi018.stdout:(105/157): lua-devel-5.4.4-4.el9.x86_64.rpm 379 kB/s | 22 kB 00:00 2024-09-21T02:47:28.709 INFO:teuthology.orchestra.run.smithi018.stdout:(106/157): abseil-cpp-20211102.0-3.el9.x86_64.r 5.8 MB/s | 548 kB 00:00 2024-09-21T02:47:28.726 INFO:teuthology.orchestra.run.smithi018.stdout:(107/157): grpc-data-1.46.7-10.el9.noarch.rpm 1.2 MB/s | 19 kB 00:00 2024-09-21T02:47:28.760 INFO:teuthology.orchestra.run.smithi018.stdout:(108/157): gperftools-libs-2.9.1-3.el9.x86_64.r 2.6 MB/s | 308 kB 00:00 2024-09-21T02:47:28.785 INFO:teuthology.orchestra.run.smithi018.stdout:(109/157): libarrow-doc-9.0.0-11.el9.noarch.rpm 1.0 MB/s | 26 kB 00:00 2024-09-21T02:47:28.810 INFO:teuthology.orchestra.run.smithi018.stdout:(110/157): protobuf-compiler-3.14.0-13.el9.x86_ 4.0 MB/s | 863 kB 00:00 2024-09-21T02:47:28.835 INFO:teuthology.orchestra.run.smithi018.stdout:(111/157): liboath-2.6.7-2.el9.x86_64.rpm 965 kB/s | 49 kB 00:00 2024-09-21T02:47:28.877 INFO:teuthology.orchestra.run.smithi018.stdout:(112/157): luarocks-3.9.2-5.el9.noarch.rpm 4.6 MB/s | 151 kB 00:00 2024-09-21T02:47:28.911 INFO:teuthology.orchestra.run.smithi018.stdout:(113/157): libunwind-1.6.2-1.el9.x86_64.rpm 670 kB/s | 67 kB 00:00 2024-09-21T02:47:28.953 INFO:teuthology.orchestra.run.smithi018.stdout:(114/157): parquet-libs-9.0.0-11.el9.x86_64.rpm 11 MB/s | 839 kB 00:00 2024-09-21T02:47:28.978 INFO:teuthology.orchestra.run.smithi018.stdout:(115/157): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.7 MB/s | 43 kB 00:00 2024-09-21T02:47:29.003 INFO:teuthology.orchestra.run.smithi018.stdout:(116/157): python3-cachetools-4.2.4-1.el9.noarc 1.3 MB/s | 32 kB 00:00 2024-09-21T02:47:29.029 INFO:teuthology.orchestra.run.smithi018.stdout:(117/157): python3-certifi-2023.05.07-4.el9.noa 566 kB/s | 14 kB 00:00 2024-09-21T02:47:29.062 INFO:teuthology.orchestra.run.smithi018.stdout:(118/157): python3-asyncssh-2.13.2-5.el9.noarch 3.5 MB/s | 548 kB 00:00 2024-09-21T02:47:29.088 INFO:teuthology.orchestra.run.smithi018.stdout:(119/157): python3-cheroot-8.6.0-4.el9.noarch.r 2.9 MB/s | 172 kB 00:00 2024-09-21T02:47:29.121 INFO:teuthology.orchestra.run.smithi018.stdout:(120/157): python3-cmd2-2.4.2-2.el9.noarch.rpm 8.9 MB/s | 303 kB 00:00 2024-09-21T02:47:29.147 INFO:teuthology.orchestra.run.smithi018.stdout:(121/157): python3-cherrypy-18.6.1-2.el9.noarch 4.2 MB/s | 358 kB 00:00 2024-09-21T02:47:29.172 INFO:teuthology.orchestra.run.smithi018.stdout:(122/157): python3-google-auth-2.34.0-1.el9.noa 4.3 MB/s | 220 kB 00:00 2024-09-21T02:47:29.197 INFO:teuthology.orchestra.run.smithi018.stdout:(123/157): python3-grpcio-tools-1.46.7-10.el9.x 5.7 MB/s | 144 kB 00:00 2024-09-21T02:47:29.222 INFO:teuthology.orchestra.run.smithi018.stdout:(124/157): python3-jaraco-8.2.1-3.el9.noarch.rp 430 kB/s | 11 kB 00:00 2024-09-21T02:47:29.248 INFO:teuthology.orchestra.run.smithi018.stdout:(125/157): python3-jaraco-classes-3.2.1-5.el9.n 707 kB/s | 18 kB 00:00 2024-09-21T02:47:29.273 INFO:teuthology.orchestra.run.smithi018.stdout:(126/157): python3-jaraco-collections-3.0.0-8.e 932 kB/s | 23 kB 00:00 2024-09-21T02:47:29.298 INFO:teuthology.orchestra.run.smithi018.stdout:(127/157): python3-jaraco-functools-3.5.0-2.el9 779 kB/s | 19 kB 00:00 2024-09-21T02:47:29.324 INFO:teuthology.orchestra.run.smithi018.stdout:(128/157): python3-jaraco-text-3.2.0-6.el9.noar 786 kB/s | 20 kB 00:00 2024-09-21T02:47:29.391 INFO:teuthology.orchestra.run.smithi018.stdout:(129/157): libarrow-9.0.0-11.el9.x86_64.rpm 6.6 MB/s | 4.4 MB 00:00 2024-09-21T02:47:29.449 INFO:teuthology.orchestra.run.smithi018.stdout:(130/157): python3-kubernetes-26.1.0-2.el9.noar 8.2 MB/s | 1.0 MB 00:00 2024-09-21T02:47:29.475 INFO:teuthology.orchestra.run.smithi018.stdout:(131/157): python3-logutils-0.3.5-21.el9.noarch 561 kB/s | 46 kB 00:00 2024-09-21T02:47:29.500 INFO:teuthology.orchestra.run.smithi018.stdout:(132/157): python3-more-itertools-8.12.0-2.el9. 1.5 MB/s | 79 kB 00:00 2024-09-21T02:47:29.525 INFO:teuthology.orchestra.run.smithi018.stdout:(133/157): python3-natsort-7.1.1-5.el9.noarch.r 1.1 MB/s | 58 kB 00:00 2024-09-21T02:47:29.575 INFO:teuthology.orchestra.run.smithi018.stdout:(134/157): python3-grpcio-1.46.7-10.el9.x86_64. 4.8 MB/s | 2.0 MB 00:00 2024-09-21T02:47:29.582 INFO:teuthology.orchestra.run.smithi016.stdout:Transaction test succeeded. 2024-09-21T02:47:29.582 INFO:teuthology.orchestra.run.smithi016.stdout:Running transaction 2024-09-21T02:47:29.601 INFO:teuthology.orchestra.run.smithi018.stdout:(135/157): python3-portend-3.1.0-2.el9.noarch.r 218 kB/s | 16 kB 00:00 2024-09-21T02:47:29.626 INFO:teuthology.orchestra.run.smithi018.stdout:(136/157): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.8 MB/s | 90 kB 00:00 2024-09-21T02:47:29.651 INFO:teuthology.orchestra.run.smithi018.stdout:(137/157): python3-pyperclip-1.8.2-2.el9.noarch 563 kB/s | 28 kB 00:00 2024-09-21T02:47:29.676 INFO:teuthology.orchestra.run.smithi018.stdout:(138/157): python3-pecan-1.4.2-3.el9.noarch.rpm 1.5 MB/s | 272 kB 00:00 2024-09-21T02:47:29.702 INFO:teuthology.orchestra.run.smithi018.stdout:(139/157): python3-repoze-lru-0.7-16.el9.noarch 615 kB/s | 31 kB 00:00 2024-09-21T02:47:29.727 INFO:teuthology.orchestra.run.smithi018.stdout:(140/157): python3-rsa-4.9-2.el9.noarch.rpm 1.2 MB/s | 59 kB 00:00 2024-09-21T02:47:29.752 INFO:teuthology.orchestra.run.smithi018.stdout:(141/157): python3-tempora-5.0.0-2.el9.noarch.r 715 kB/s | 36 kB 00:00 2024-09-21T02:47:29.778 INFO:teuthology.orchestra.run.smithi018.stdout:(142/157): python3-routes-2.5.1-5.el9.noarch.rp 1.5 MB/s | 188 kB 00:00 2024-09-21T02:47:29.803 INFO:teuthology.orchestra.run.smithi018.stdout:(143/157): python3-typing-extensions-4.12.2-2.e 983 kB/s | 74 kB 00:00 2024-09-21T02:47:29.820 INFO:teuthology.orchestra.run.smithi018.stdout:(144/157): python3-websocket-client-1.2.3-2.el9 2.1 MB/s | 90 kB 00:00 2024-09-21T02:47:29.845 INFO:teuthology.orchestra.run.smithi018.stdout:(145/157): python3-webob-1.8.8-2.el9.noarch.rpm 2.4 MB/s | 230 kB 00:00 2024-09-21T02:47:29.862 INFO:teuthology.orchestra.run.smithi018.stdout:(146/157): python3-zc-lockfile-2.0-10.el9.noarc 477 kB/s | 20 kB 00:00 2024-09-21T02:47:29.887 INFO:teuthology.orchestra.run.smithi018.stdout:(147/157): re2-20211101-20.el9.x86_64.rpm 4.5 MB/s | 191 kB 00:00 2024-09-21T02:47:29.913 INFO:teuthology.orchestra.run.smithi018.stdout:(148/157): python3-werkzeug-2.0.3-3.el9.1.noarc 3.8 MB/s | 427 kB 00:00 2024-09-21T02:47:29.963 INFO:teuthology.orchestra.run.smithi018.stdout:(149/157): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-09-21T02:47:30.138 INFO:teuthology.orchestra.run.smithi018.stdout:(150/157): librados2-19.3.0-5120.g18432218.el9. 14 MB/s | 3.4 MB 00:00 2024-09-21T02:47:30.188 INFO:teuthology.orchestra.run.smithi018.stdout:(151/157): NetworkManager-1.51.0-1.el9.x86_64.r 10 MB/s | 2.3 MB 00:00 2024-09-21T02:47:30.230 INFO:teuthology.orchestra.run.smithi018.stdout:(152/157): NetworkManager-team-1.51.0-1.el9.x86 989 kB/s | 41 kB 00:00 2024-09-21T02:47:30.272 INFO:teuthology.orchestra.run.smithi018.stdout:(153/157): NetworkManager-tui-1.51.0-1.el9.x86_ 5.8 MB/s | 249 kB 00:00 2024-09-21T02:47:30.331 INFO:teuthology.orchestra.run.smithi018.stdout:(154/157): NetworkManager-libnm-1.51.0-1.el9.x8 9.4 MB/s | 1.8 MB 00:00 2024-09-21T02:47:30.373 INFO:teuthology.orchestra.run.smithi018.stdout:(155/157): openssl-3.2.2-6.el9.x86_64.rpm 14 MB/s | 1.4 MB 00:00 2024-09-21T02:47:30.423 INFO:teuthology.orchestra.run.smithi018.stdout:(156/157): librbd1-19.3.0-5120.g18432218.el9.x8 6.2 MB/s | 3.2 MB 00:00 2024-09-21T02:47:30.473 INFO:teuthology.orchestra.run.smithi018.stdout:(157/157): openssl-libs-3.2.2-6.el9.x86_64.rpm 17 MB/s | 2.4 MB 00:00 2024-09-21T02:47:30.480 INFO:teuthology.orchestra.run.smithi018.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:47:30.480 INFO:teuthology.orchestra.run.smithi018.stdout:Total 19 MB/s | 228 MB 00:11 2024-09-21T02:47:31.742 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction check 2024-09-21T02:47:31.912 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction check succeeded. 2024-09-21T02:47:31.912 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction test 2024-09-21T02:47:33.835 INFO:teuthology.orchestra.run.smithi018.stdout:Transaction test succeeded. 2024-09-21T02:47:33.835 INFO:teuthology.orchestra.run.smithi018.stdout:Running transaction 2024-09-21T02:47:34.380 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2024-09-21T02:47:34.487 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:47:34.525 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:47:34.596 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:47:34.646 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:47:34.672 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:47:34.713 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:47:35.052 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:47:35.173 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:35.281 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:35.315 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:35.406 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:35.436 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:47:35.465 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:47:35.616 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:47:35.667 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:47:35.703 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:35.808 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:35.846 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:35.908 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:35.938 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:35.961 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:36.157 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:36.225 INFO:teuthology.orchestra.run.smithi016.stdout: Preparing : 1/1 2024-09-21T02:47:36.298 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:36.347 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:47:36.410 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:47:36.428 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:47:36.524 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:36.608 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:36.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:47:36.657 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:47:36.682 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:47:36.746 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:47:36.851 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:36.966 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:47:36.995 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:47:37.030 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:47:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:47:37.120 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:47:37.172 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:47:37.204 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:47:37.232 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:37.248 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:47:37.280 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:47:37.309 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:47:37.346 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:37.375 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:37.376 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:47:37.430 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:47:37.451 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:37.469 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:47:37.489 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:47:37.504 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:47:37.519 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:47:37.585 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:47:37.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:47:37.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:47:37.688 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:47:37.703 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:47:37.756 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:47:37.762 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:47:37.797 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:37.801 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:47:37.876 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:47:37.905 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:37.940 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:38.016 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:47:38.028 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:38.058 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:38.070 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:47:38.085 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:38.288 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:38.492 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:38.587 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:47:38.615 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:47:38.674 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:47:38.712 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:47:38.720 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:38.762 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:47:38.801 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:47:38.808 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:38.842 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:47:39.012 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:47:39.051 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:39.070 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:47:39.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:47:39.177 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:47:39.216 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:47:39.234 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:47:39.256 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:47:39.316 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:47:39.332 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:47:39.347 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:47:39.383 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:47:39.383 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:47:39.423 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:47:39.430 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:47:39.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:47:39.466 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:47:39.499 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:47:39.509 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:47:39.535 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:47:39.551 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:47:39.615 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:47:39.677 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:47:39.685 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:47:39.705 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:47:39.728 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:47:39.749 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:47:39.832 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:47:39.867 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:47:39.912 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:47:39.940 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:47:40.006 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:47:40.013 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:47:40.072 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:47:40.119 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:47:40.153 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:47:40.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:47:40.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:47:40.218 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:47:40.243 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:47:40.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:47:40.319 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:47:40.367 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:47:40.444 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:47:40.482 INFO:teuthology.orchestra.run.smithi018.stdout: Preparing : 1/1 2024-09-21T02:47:40.485 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:47:40.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:47:40.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:47:40.576 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-libs-1:3.2.2-6.el9.x86_64 1/165 2024-09-21T02:47:40.594 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:47:40.616 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 2/165 2024-09-21T02:47:40.635 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:47:40.664 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:47:40.699 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:47:40.732 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:47:40.774 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:47:40.808 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:47:40.849 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 3/165 2024-09-21T02:47:40.888 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : thrift-0.15.0-2.el9.x86_64 4/165 2024-09-21T02:47:40.895 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:47:40.920 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 5/165 2024-09-21T02:47:40.926 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:47:40.939 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:47:40.966 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 6/165 2024-09-21T02:47:40.978 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:47:41.022 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:47:41.057 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:47:41.081 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:47:41.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:47:41.137 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:47:41.138 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:47:41.181 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:47:41.325 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 7/165 2024-09-21T02:47:41.351 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:47:41.408 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:47:41.446 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:41.504 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:47:41.558 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:47:41.566 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:47:41.588 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:41.664 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:47:41.673 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:47:41.693 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:47:41.711 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:47:41.722 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-private-libs-13.16-1.el9.x86_64 11/165 2024-09-21T02:47:41.747 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:47:41.794 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:47:41.838 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:47:41.855 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:47:41.887 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libnbd-1.20.2-2.el9.x86_64 12/165 2024-09-21T02:47:41.890 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:47:41.932 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:47:41.941 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-13.16-1.el9.x86_64 13/165 2024-09-21T02:47:41.968 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:42.060 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:47:42.076 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:47:42.086 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:42.129 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:47:42.143 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:47:42.149 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 15/165 2024-09-21T02:47:42.171 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:42.171 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:47:42.177 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:47:42.193 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 16/165 2024-09-21T02:47:42.276 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:47:42.278 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:47:42.312 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:47:42.332 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:47:42.359 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:47:42.376 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:47:42.389 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:42.394 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:47:42.419 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:47:42.464 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:47:42.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:47:42.517 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 17/165 2024-09-21T02:47:42.531 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:47:42.635 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 18/165 2024-09-21T02:47:42.659 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:47:42.693 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:47:42.697 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:47:42.717 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:42.721 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:47:42.727 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:47:42.764 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:47:42.811 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:42.812 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:47:42.812 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:47:42.852 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:47:42.882 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:47:42.916 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:47:42.950 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:47:42.992 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:47:43.026 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:47:43.048 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.51.0-1.el9.x86_64 19/165 2024-09-21T02:47:43.117 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:47:43.155 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : re2-1:20211101-20.el9.x86_64 20/165 2024-09-21T02:47:43.182 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 21/165 2024-09-21T02:47:43.218 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : liboath-2.6.7-2.el9.x86_64 22/165 2024-09-21T02:47:43.274 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:47:43.293 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 23/165 2024-09-21T02:47:43.313 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:47:43.344 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 24/165 2024-09-21T02:47:43.384 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-packaging-20.9-5.el9.noarch 25/165 2024-09-21T02:47:43.421 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:47:43.428 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 26/165 2024-09-21T02:47:43.446 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:47:43.469 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 27/165 2024-09-21T02:47:43.485 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:47:43.496 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-5.4.4-4.el9.x86_64 28/165 2024-09-21T02:47:43.558 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:47:43.568 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 29/165 2024-09-21T02:47:43.621 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-urllib3-1.26.5-6.el9.noarch 30/165 2024-09-21T02:47:43.649 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 31/165 2024-09-21T02:47:43.693 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libquadmath-11.5.0-2.el9.x86_64 32/165 2024-09-21T02:47:43.781 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libgfortran-11.5.0-2.el9.x86_64 33/165 2024-09-21T02:47:43.816 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 34/165 2024-09-21T02:47:43.859 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 35/165 2024-09-21T02:47:43.888 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 36/165 2024-09-21T02:47:43.963 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 37/165 2024-09-21T02:47:44.012 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : luarocks-3.9.2-5.el9.noarch 38/165 2024-09-21T02:47:44.088 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 39/165 2024-09-21T02:47:44.234 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 40/165 2024-09-21T02:47:44.269 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:47:44.298 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 41/165 2024-09-21T02:47:44.314 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:47:44.348 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:47:44.454 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:47:44.513 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:47:44.566 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:47:44.666 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:47:44.710 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:47:44.774 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:47:44.847 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:47:44.857 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rsa-4.9-2.el9.noarch 42/165 2024-09-21T02:47:44.875 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:47:44.902 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:47:44.908 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:47:44.932 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:47:44.945 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 43/165 2024-09-21T02:47:44.988 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jinja2-2.11.3-6.el9.noarch 44/165 2024-09-21T02:47:45.012 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:47:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 45/165 2024-09-21T02:47:45.070 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 46/165 2024-09-21T02:47:45.111 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 47/165 2024-09-21T02:47:45.233 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:47:45.278 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : socat-1.7.4.1-6.el9.x86_64 48/165 2024-09-21T02:47:45.279 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:45.331 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : openssl-1:3.2.2-6.el9.x86_64 49/165 2024-09-21T02:47:45.370 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:45.427 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 50/165 2024-09-21T02:47:45.497 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 51/165 2024-09-21T02:47:45.500 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:47:45.536 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:47:45.571 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 52/165 2024-09-21T02:47:45.607 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-webob-1.8.8-2.el9.noarch 53/165 2024-09-21T02:47:45.644 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-typing-extensions-4.12.2-2.el9.noarch 54/165 2024-09-21T02:47:45.692 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 55/165 2024-09-21T02:47:45.704 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:47:45.729 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 56/165 2024-09-21T02:47:45.772 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 57/165 2024-09-21T02:47:45.814 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/165 2024-09-21T02:47:45.826 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:47:45.958 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 59/165 2024-09-21T02:47:46.033 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:47:46.074 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:46.099 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 60/165 2024-09-21T02:47:46.144 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:46.149 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 61/165 2024-09-21T02:47:46.175 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:47:46.257 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 62/165 2024-09-21T02:47:46.290 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-google-auth-1:2.34.0-1.el9.noarch 63/165 2024-09-21T02:47:46.331 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 64/165 2024-09-21T02:47:46.356 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 65/165 2024-09-21T02:47:46.381 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 66/165 2024-09-21T02:47:46.459 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 67/165 2024-09-21T02:47:46.588 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 68/165 2024-09-21T02:47:46.621 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 69/165 2024-09-21T02:47:46.649 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 70/165 2024-09-21T02:47:46.684 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 71/165 2024-09-21T02:47:46.723 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : uuid-1.6.2-55.el9.x86_64 72/165 2024-09-21T02:47:46.763 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 73/165 2024-09-21T02:47:46.792 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 74/165 2024-09-21T02:47:46.827 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 75/165 2024-09-21T02:47:46.860 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 76/165 2024-09-21T02:47:46.903 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 77/165 2024-09-21T02:47:46.936 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 78/165 2024-09-21T02:47:47.027 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 79/165 2024-09-21T02:47:47.076 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 80/165 2024-09-21T02:47:47.106 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 81/165 2024-09-21T02:47:47.158 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:47:47.214 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 82/165 2024-09-21T02:47:47.229 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:47:47.257 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:47:47.267 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 83/165 2024-09-21T02:47:47.290 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:47:47.641 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:47:47.703 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:47.779 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:47.808 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:47:47.844 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:47:48.032 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 84/165 2024-09-21T02:47:48.081 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 85/165 2024-09-21T02:47:48.115 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 86/165 2024-09-21T02:47:48.219 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 87/165 2024-09-21T02:47:48.271 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-devel-3.9.19-8.el9.x86_64 88/165 2024-09-21T02:47:48.316 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-colorama-0.4.6-3.el9.noarch 89/165 2024-09-21T02:47:48.379 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:47:48.406 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 90/165 2024-09-21T02:47:48.406 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:48.440 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:48.443 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 91/165 2024-09-21T02:47:48.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:48.477 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:48.508 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:47:48.672 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-0.3.26-2.el9.x86_64 92/165 2024-09-21T02:47:48.798 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openblas-openmp-0.3.26-2.el9.x86_64 93/165 2024-09-21T02:47:48.879 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 94/165 2024-09-21T02:47:49.040 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:49.083 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:49.170 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:49.170 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:47:49.171 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:47:49.171 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:47:49.201 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:47:49.560 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 95/165 2024-09-21T02:47:49.695 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 96/165 2024-09-21T02:47:50.999 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 97/165 2024-09-21T02:47:51.186 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 98/165 2024-09-21T02:47:51.215 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 99/165 2024-09-21T02:47:51.247 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libuuid-devel-2.37.4-20.el9.x86_64 100/165 2024-09-21T02:47:51.294 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:51.331 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:51.591 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 101/165 2024-09-21T02:47:51.643 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:51.744 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 102/165 2024-09-21T02:47:51.773 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 103/165 2024-09-21T02:47:51.809 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 104/165 2024-09-21T02:47:51.939 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:51.976 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:52.050 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:52.050 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:47:52.050 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:47:52.051 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:47:52.094 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:47:52.337 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 105/165 2024-09-21T02:47:52.363 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:52.434 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 106/165 2024-09-21T02:47:52.464 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 107/165 2024-09-21T02:47:54.769 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:54.811 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:55.411 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 108/165 2024-09-21T02:47:55.456 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:55.532 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 109/165 2024-09-21T02:47:55.533 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-09-21T02:47:55.533 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-09-21T02:47:55.533 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:47:55.565 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /root 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2024-09-21T02:48:00.919 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:01.094 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:01.111 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:01.111 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:01.111 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:48:01.111 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:02.705 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:02.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:02.956 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:02.979 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:02.979 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:02.979 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:48:02.980 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:02.980 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:02.980 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:03.440 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:03.463 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:03.463 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:03.464 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:48:03.464 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:03.464 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:03.464 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:03.570 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:48:03.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:48:03.681 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:48:03.875 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:48:03.923 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:48:03.993 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:48:04.081 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:48:04.220 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:48:04.265 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:48:04.305 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:48:04.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /sys 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /proc 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /mnt 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /var/tmp 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /home 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /root 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout:skipping the directory /tmp 2024-09-21T02:48:04.608 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:04.759 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:04.799 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:04.800 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:04.817 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:04.817 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:48:04.817 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:04.830 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:48:04.895 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:48:04.921 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:06.470 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:06.498 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:06.730 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:06.754 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:07.247 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:07.272 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:07.386 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:48:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:48:07.508 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:48:07.703 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:48:07.757 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:48:07.838 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:48:07.936 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:48:08.100 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 110/165 2024-09-21T02:48:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /sys 2024-09-21T02:48:08.101 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /proc 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /mnt 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /var/tmp 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /home 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /root 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout:skipping the directory /tmp 2024-09-21T02:48:08.102 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:08.105 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:48:08.106 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:08.149 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:48:08.177 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:08.180 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:48:08.208 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:48:08.277 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:08.293 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 111/165 2024-09-21T02:48:08.293 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:08.293 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-09-21T02:48:08.293 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:08.645 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:08.674 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:08.715 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:48:08.784 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:48:08.814 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:09.134 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:09.161 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:09.274 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:09.443 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:48:09.471 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:09.496 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:09.547 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:09.558 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:09.657 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:48:09.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:48:09.955 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:09.982 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 112/165 2024-09-21T02:48:09.982 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:09.982 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-09-21T02:48:09.982 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:09.982 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-09-21T02:48:09.983 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:10.214 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 113/165 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-09-21T02:48:10.238 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:10.681 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:48:10.716 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:10.739 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:10.739 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:10.739 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:48:10.739 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:10.740 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:10.740 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:10.740 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 114/165 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-09-21T02:48:10.764 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:10.783 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:10.806 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:10.806 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:48:10.806 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:10.850 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:10.871 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : automake-1.16.2-8.el9.noarch 115/165 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:10.874 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:10.929 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-ply-3.11-14.el9.noarch 116/165 2024-09-21T02:48:10.993 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 117/165 2024-09-21T02:48:11.186 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 118/165 2024-09-21T02:48:11.193 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:11.216 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:48:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:11.217 INFO:teuthology.orchestra.run.smithi123.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:11.217 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:11.233 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 119/165 2024-09-21T02:48:11.314 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 120/165 2024-09-21T02:48:11.413 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 121/165 2024-09-21T02:48:11.563 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 122/165 2024-09-21T02:48:11.600 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 123/165 2024-09-21T02:48:11.632 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 124/165 2024-09-21T02:48:11.659 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : fuse-2.9.9-16.el9.x86_64 125/165 2024-09-21T02:48:12.097 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:12.125 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephadm-2:19.3.0-5120.g18432218.el9.noarch 126/165 2024-09-21T02:48:12.158 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 127/165 2024-09-21T02:48:12.227 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 128/165 2024-09-21T02:48:12.227 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:12.264 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:12.287 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:13.302 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:13.337 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:13.459 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:13.637 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:48:13.673 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:13.698 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:13.748 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:13.760 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:13.863 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:48:13.923 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:48:14.941 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:48:14.968 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:14.992 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:14.993 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:14.993 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:48:14.993 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:14.993 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:14.993 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:15.035 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:15.059 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:15.059 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:15.059 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:48:15.059 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:15.111 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:15.135 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:15.135 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:15.135 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:48:15.135 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:15.136 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:15.136 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:15.172 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:48:15.213 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:48:15.254 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:48:15.294 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:48:15.346 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:48:15.478 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:15.502 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:15.502 INFO:teuthology.orchestra.run.smithi016.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:48:15.503 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:15.503 INFO:teuthology.orchestra.run.smithi016.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:15.503 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:15.538 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:48:15.584 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:48:15.615 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:48:15.644 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:48:15.650 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 129/165 2024-09-21T02:48:15.691 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:48:15.703 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:16.317 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:16.355 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:16.376 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:16.721 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 130/165 2024-09-21T02:48:16.745 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:16.866 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 131/165 2024-09-21T02:48:17.038 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 132/165 2024-09-21T02:48:17.064 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 133/165 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-09-21T02:48:17.088 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:17.140 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:17.151 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 134/165 2024-09-21T02:48:17.209 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:48:17.255 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-2:19.3.0-5120.g18432218.el9.x86_64 135/165 2024-09-21T02:48:17.293 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:48:17.306 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 136/165 2024-09-21T02:48:17.334 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:48:17.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:48:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:48:17.479 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:48:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:48:17.505 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:17.581 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:17.582 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:17.674 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:17.699 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:48:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:17.704 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:17.884 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:17.959 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:18.022 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:18.318 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libtool-2.4.6-46.el9.x86_64 137/165 2024-09-21T02:48:18.351 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:18.373 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 138/165 2024-09-21T02:48:18.373 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:18.374 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-09-21T02:48:18.374 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:18.374 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-09-21T02:48:18.374 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:18.418 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:18.440 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.3.0-5120.g18432 139/165 2024-09-21T02:48:18.441 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:18.441 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-09-21T02:48:18.441 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:18.502 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 140/165 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-09-21T02:48:18.525 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:18.862 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 141/165 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-09-21T02:48:18.884 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:19.770 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:48:19.816 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:48:19.858 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:48:19.889 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:48:19.916 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:48:20.156 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:48:20.205 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:48:20.235 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:48:20.264 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:48:20.310 INFO:teuthology.orchestra.run.smithi016.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:48:20.969 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:21.017 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:21.037 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:21.929 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:48:22.030 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:48:22.080 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:48:22.175 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:48:22.209 INFO:teuthology.orchestra.run.smithi016.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:48:22.234 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:48:22.268 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:48:22.269 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:22.353 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:22.353 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:22.437 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:22.461 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:48:22.467 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:22.467 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:22.655 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:22.655 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:22.748 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:22.825 INFO:teuthology.orchestra.run.smithi016.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:23.099 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 142/165 2024-09-21T02:48:23.141 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 143/165 2024-09-21T02:48:23.181 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 144/165 2024-09-21T02:48:23.212 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : xfsprogs-devel-6.4.0-4.el9.x86_64 145/165 2024-09-21T02:48:23.231 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : elfutils-libelf-devel-0.191-4.el9.x86_64 146/165 2024-09-21T02:48:23.416 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 147/165 2024-09-21T02:48:23.462 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-contrib-13.16-1.el9.x86_64 148/165 2024-09-21T02:48:23.492 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 149/165 2024-09-21T02:48:23.522 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-team-1:1.51.0-1.el9.x86_64 150/165 2024-09-21T02:48:23.560 INFO:teuthology.orchestra.run.smithi018.stdout: Upgrading : NetworkManager-tui-1:1.51.0-1.el9.x86_64 151/165 2024-09-21T02:48:24.164 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:24.217 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:24.238 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: postgresql-server-13.16-1.el9.x86_64 152/165 2024-09-21T02:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:48:24.258 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:48:24.259 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:48:24.260 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:48:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:48:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:48:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:48:24.261 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:48:24.275 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:48:24.276 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:48:24.277 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:48:24.278 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:48:24.279 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:48:24.280 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:48:24.281 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:48:24.282 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:48:24.283 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:48:24.284 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:48:25.137 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 153/165 2024-09-21T02:48:25.229 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : openssl-devel-1:3.2.2-6.el9.x86_64 154/165 2024-09-21T02:48:25.271 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 155/165 2024-09-21T02:48:25.355 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : flex-2.6.4-9.el9.x86_64 156/165 2024-09-21T02:48:25.382 INFO:teuthology.orchestra.run.smithi018.stdout: Installing : bison-3.7.4-5.el9.x86_64 157/165 2024-09-21T02:48:25.407 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-tui-1:1.48.8-1.el9.x86_64 158/165 2024-09-21T02:48:25.433 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-1:3.2.2-4.el9.x86_64 159/165 2024-09-21T02:48:25.433 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:25.518 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 160/165 2024-09-21T02:48:25.518 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:25.601 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 161/165 2024-09-21T02:48:25.626 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-team-1:1.48.8-1.el9.x86_64 162/165 2024-09-21T02:48:25.632 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:25.632 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:25.812 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-1:1.48.8-1.el9.x86_64 163/165 2024-09-21T02:48:25.812 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:25.912 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: NetworkManager-libnm-1:1.48.8-1.el9.x86_64 164/165 2024-09-21T02:48:25.979 INFO:teuthology.orchestra.run.smithi018.stdout: Cleanup : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.791 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.792 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.793 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:48:26.794 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:48:26.795 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:48:26.796 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:48:26.797 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:48:26.798 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:48:26.799 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.800 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:48:26.801 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:48:26.802 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:48:26.803 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:48:26.803 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2024-09-21T02:48:27.227 DEBUG:teuthology.parallel:result is None 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:48:29.524 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:48:29.525 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:48:29.526 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:48:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:48:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:48:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:48:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:48:29.537 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:48:29.538 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:48:29.539 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:48:29.540 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:48:29.541 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:48:29.542 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:48:29.543 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:48:29.544 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:48:29.545 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:48:29.546 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout:Upgraded: 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:31.959 INFO:teuthology.orchestra.run.smithi016.stdout:Installed: 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.960 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.961 INFO:teuthology.orchestra.run.smithi016.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.962 INFO:teuthology.orchestra.run.smithi016.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:48:31.963 INFO:teuthology.orchestra.run.smithi016.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:48:31.964 INFO:teuthology.orchestra.run.smithi016.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:48:31.965 INFO:teuthology.orchestra.run.smithi016.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:48:31.966 INFO:teuthology.orchestra.run.smithi016.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:48:31.967 INFO:teuthology.orchestra.run.smithi016.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:48:31.968 INFO:teuthology.orchestra.run.smithi016.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:48:31.969 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:48:31.970 INFO:teuthology.orchestra.run.smithi016.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:48:31.971 INFO:teuthology.orchestra.run.smithi016.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:48:31.971 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:48:31.971 INFO:teuthology.orchestra.run.smithi016.stdout:Complete! 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Running scriptlet: openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-2:19.3.0-5120.g18432218.el9.x86_64 1/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 3/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 4/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-immutable-object-cache-2:19.3.0-5120.g18432 5/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 6/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 7/165 2024-09-21T02:48:32.040 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 8/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 9/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 10/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 11/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 12/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 13/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 14/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_ 15/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 16/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 17/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados-devel-2:19.3.0-5120.g18432218.el9.x86_6 18/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libradosstriper1-2:19.3.0-5120.g18432218.el9.x86 19/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librgw2-2:19.3.0-5120.g18432218.el9.x86_64 20/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-argparse-2:19.3.0-5120.g18432218.el 21/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ceph-common-2:19.3.0-5120.g18432218.el9. 22/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_6 23/165 2024-09-21T02:48:32.041 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 24/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 25/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 26/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 27/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 28/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 29/165 2024-09-21T02:48:32.042 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-grafana-dashboards-2:19.3.0-5120.g18432218. 30/165 2024-09-21T02:48:32.053 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noa 31/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.n 32/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.3.0-5120.g184 33/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el 34/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 35/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-prometheus-alerts-2:19.3.0-5120.g18432218.e 36/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 37/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephadm-2:19.3.0-5120.g18432218.el9.noarch 38/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 39/165 2024-09-21T02:48:32.054 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : fuse-2.9.9-16.el9.x86_64 40/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libgfortran-11.5.0-2.el9.x86_64 41/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libquadmath-11.5.0-2.el9.x86_64 42/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-urllib3-1.26.5-6.el9.noarch 48/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : elfutils-libelf-devel-0.191-4.el9.x86_64 52/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/165 2024-09-21T02:48:32.055 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libnbd-1.20.2-2.el9.x86_64 58/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 59/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 60/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 61/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libtool-2.4.6-46.el9.x86_64 62/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libuuid-devel-2.37.4-20.el9.x86_64 63/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 64/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 65/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-5.4.4-4.el9.x86_64 66/165 2024-09-21T02:48:32.056 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-0.3.26-2.el9.x86_64 67/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openblas-openmp-0.3.26-2.el9.x86_64 68/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-devel-1:3.2.2-6.el9.x86_64 69/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-13.16-1.el9.x86_64 70/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-contrib-13.16-1.el9.x86_64 71/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-private-libs-13.16-1.el9.x86_64 72/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : postgresql-server-13.16-1.el9.x86_64 73/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 74/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 75/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 76/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-colorama-0.4.6-3.el9.noarch 77/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-devel-3.9.19-8.el9.x86_64 78/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jinja2-2.11.3-6.el9.noarch 79/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 80/165 2024-09-21T02:48:32.057 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 81/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 82/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 84/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 85/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 86/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 87/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 88/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 89/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 92/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/165 2024-09-21T02:48:32.058 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 94/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 96/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 97/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 98/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 99/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : socat-1.7.4.1-6.el9.x86_64 100/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 101/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xfsprogs-devel-6.4.0-4.el9.x86_64 102/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 103/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 104/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 105/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 106/165 2024-09-21T02:48:32.059 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 107/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 108/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 109/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 110/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 111/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 112/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 113/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 114/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 115/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 116/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 117/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 118/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 119/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 120/165 2024-09-21T02:48:32.060 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 121/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 122/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-google-auth-1:2.34.0-1.el9.noarch 123/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 124/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 125/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 126/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 127/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 128/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 129/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 130/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 131/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 132/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 133/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 134/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 135/165 2024-09-21T02:48:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 136/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 137/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 138/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 139/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 140/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 141/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 142/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-typing-extensions-4.12.2-2.el9.noarch 143/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-webob-1.8.8-2.el9.noarch 144/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 145/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 146/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 147/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : re2-1:20211101-20.el9.x86_64 148/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 149/165 2024-09-21T02:48:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:19.3.0-5120.g18432218.el9.x86_64 150/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 151/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:19.3.0-5120.g18432218.el9.x86_64 152/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 153/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.51.0-1.el9.x86_64 154/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-1:1.48.8-1.el9.x86_64 155/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.51.0-1.el9.x86_64 156/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-libnm-1:1.48.8-1.el9.x86_64 157/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.51.0-1.el9.x86_64 158/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-team-1:1.48.8-1.el9.x86_64 159/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.51.0-1.el9.x86_64 160/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : NetworkManager-tui-1:1.48.8-1.el9.x86_64 161/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.2.2-6.el9.x86_64 162/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-1:3.2.2-4.el9.x86_64 163/165 2024-09-21T02:48:32.063 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.2.2-6.el9.x86_64 164/165 2024-09-21T02:48:32.545 DEBUG:teuthology.parallel:result is None 2024-09-21T02:48:34.521 INFO:teuthology.orchestra.run.smithi018.stdout: Verifying : openssl-libs-1:3.2.2-4.el9.x86_64 165/165 2024-09-21T02:48:34.521 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:34.521 INFO:teuthology.orchestra.run.smithi018.stdout:Upgraded: 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-libnm-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-team-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: NetworkManager-tui-1:1.51.0-1.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: librados2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: librbd1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-libs-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout:Installed: 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: automake-1.16.2-8.el9.noarch 2024-09-21T02:48:34.522 INFO:teuthology.orchestra.run.smithi018.stdout: bison-3.7.4-5.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-base-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-grafana-dashboards-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-immutable-object-cache-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mds-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-dashboard-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-diskprediction-local-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-modules-core-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.523 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mgr-rook-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-mon-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-osd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-prometheus-alerts-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-radosgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-selinux-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-test-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: ceph-volume-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: cephadm-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-shell-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: cephfs-top-2:19.3.0-5120.g18432218.el9.noarch 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: elfutils-libelf-devel-0.191-4.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: flex-2.6.4-9.el9.x86_64 2024-09-21T02:48:34.524 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: fuse-2.9.9-16.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: grpc-data-1.46.7-10.el9.noarch 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-9.0.0-11.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libcephfs2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libcephsqlite-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libgfortran-11.5.0-2.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: libnbd-1.20.2-2.el9.x86_64 2024-09-21T02:48:34.525 INFO:teuthology.orchestra.run.smithi018.stdout: liboath-2.6.7-2.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libquadmath-11.5.0-2.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: librados-devel-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libradosstriper1-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: librgw2-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libtool-2.4.6-46.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libunwind-1.6.2-1.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libuuid-devel-2.37.4-20.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-09-21T02:48:34.526 INFO:teuthology.orchestra.run.smithi018.stdout: lua-5.4.4-4.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: luarocks-3.9.2-5.el9.noarch 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-0.3.26-2.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: openblas-openmp-0.3.26-2.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: openssl-devel-1:3.2.2-6.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-13.16-1.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-contrib-13.16-1.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-private-libs-13.16-1.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: postgresql-server-13.16-1.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-3.14.0-13.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-09-21T02:48:34.527 INFO:teuthology.orchestra.run.smithi018.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-babel-2.9.1-2.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-argparse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ceph-common-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cephfs-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-certifi-2023.05.07-4.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-colorama-0.4.6-3.el9.noarch 2024-09-21T02:48:34.528 INFO:teuthology.orchestra.run.smithi018.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-devel-3.9.19-8.el9.x86_64 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-google-auth-1:2.34.0-1.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jinja2-2.11.3-6.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-09-21T02:48:34.529 INFO:teuthology.orchestra.run.smithi018.stdout: python3-mako-1.1.4-6.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-packaging-20.9-5.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-ply-3.11-14.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-portend-3.1.0-2.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-09-21T02:48:34.530 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pycparser-2.20-6.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-pytz-2021.1-5.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rados-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-2.25.1-8.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rgw-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-routes-2.5.1-5.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-rsa-4.9-2.el9.noarch 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-09-21T02:48:34.531 INFO:teuthology.orchestra.run.smithi018.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-toml-0.10.2-6.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-typing-extensions-4.12.2-2.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-urllib3-1.26.5-6.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-webob-1.8.8-2.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-24.02.0-1.el9.x86_64 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-fuse-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.532 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-mirror-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: rbd-nbd-2:19.3.0-5120.g18432218.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: re2-1:20211101-20.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: socat-1.7.4.1-6.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: thrift-0.15.0-2.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: uuid-1.6.2-55.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: xfsprogs-devel-6.4.0-4.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:48:34.533 INFO:teuthology.orchestra.run.smithi018.stdout:Complete! 2024-09-21T02:48:34.904 DEBUG:teuthology.parallel:result is None 2024-09-21T02:48:34.905 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:35.069 DEBUG:teuthology.orchestra.run.smithi016:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:48:35.096 INFO:teuthology.orchestra.run.smithi016.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.096 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.097 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:48:35.100 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:35.243 DEBUG:teuthology.orchestra.run.smithi018:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:48:35.268 INFO:teuthology.orchestra.run.smithi018.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.268 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.268 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:48:35.270 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:35.414 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-09-21T02:48:35.440 INFO:teuthology.orchestra.run.smithi123.stdout:19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.440 INFO:teuthology.packaging:The installed version of ceph is 19.3.0-5120.g18432218.el9 2024-09-21T02:48:35.441 INFO:teuthology.task.install:The correct ceph version 19.3.0-5120.g18432218 is installed. 2024-09-21T02:48:35.442 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-09-21T02:48:35.442 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:48:35.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:48:35.477 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:48:35.477 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:48:35.509 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:48:35.509 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-09-21T02:48:35.543 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-09-21T02:48:35.543 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:48:35.543 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:48:35.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:48:35.642 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:48:35.642 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:48:35.672 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:48:35.741 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:48:35.741 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2024-09-21T02:48:35.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-09-21T02:48:35.842 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-09-21T02:48:35.842 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:48:35.842 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:48:35.873 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:48:35.943 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:48:35.943 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:48:35.972 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:48:36.039 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:48:36.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2024-09-21T02:48:36.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-09-21T02:48:36.136 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-09-21T02:48:36.137 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:48:36.137 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:48:36.168 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:48:36.238 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:48:36.238 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:48:36.267 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:48:36.337 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:48:36.337 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2024-09-21T02:48:36.366 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-09-21T02:48:36.433 INFO:teuthology.run_tasks:Running task cephadm... 2024-09-21T02:48:36.536 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True}, 'cluster-conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 2, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'client mount timeout': 30, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1, 'mgr/snap_schedule/allow_m_granularity': True, 'mgr/snap_schedule/dump_on_update': True, 'mgr/snap_schedule/log_level': 'debug', 'mon warn on pool no app': False}, 'mon': {'mon op complaint time': 120}, 'osd': {'osd op complaint time': 180}}, 'conf': {'global': {'ms die on skipped message': False}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'fs.*is degraded', 'filesystem is degraded', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'filesystem is offline', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'insufficient standby MDS daemons available', 'MDS_UP_LESS_THAN_MAX', 'online, but wants', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'Degraded data redundancy', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'mgr-modules': ['snap_schedule'], 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'subvols': {'create': 2}} 2024-09-21T02:48:36.536 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:36.537 INFO:tasks.cephadm:Cluster fsid is ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:48:36.537 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-09-21T02:48:36.537 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.16', 'mon.b': '172.21.15.18', 'mon.c': '172.21.15.123'} 2024-09-21T02:48:36.537 INFO:tasks.cephadm:First mon is mon.a on smithi016 2024-09-21T02:48:36.537 INFO:tasks.cephadm:First mgr is x 2024-09-21T02:48:36.537 INFO:tasks.cephadm:Normalizing hostnames... 2024-09-21T02:48:36.537 DEBUG:teuthology.orchestra.run.smithi016:> sudo hostname $(hostname -s) 2024-09-21T02:48:36.569 DEBUG:teuthology.orchestra.run.smithi018:> sudo hostname $(hostname -s) 2024-09-21T02:48:36.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo hostname $(hostname -s) 2024-09-21T02:48:36.627 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-09-21T02:48:36.628 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:36.765 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.4.75+confusa13', 'job_name': 'ceph-dev-new-build/ARCH=arm64,AVAILABLE_ARCH=arm64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://4.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-21 00:04:17.484514', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://4.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['x86_64', 'source', 'arm64'], 'distro': 'centos'}, {'status': 'ready', 'sha1': '1843221895adccfb3b692060251dd6a0183bc410', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/83190/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.3.0-5120-g18432218', 'node_name': '172.21.2.22+braggi22', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.3.0-5120.g18432218'}, 'url': 'https://3.chacra.ceph.com/r/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-09-20 22:54:16.499090', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-pdonnell-testing-20240920.212106-debug', 'chacra_url': 'https://3.chacra.ceph.com/repos/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-09-21T02:48:36.902 INFO:tasks.util.chacra:got chacra host 4.chacra.ceph.com, ref wip-pdonnell-testing-20240920.212106-debug, sha1 1843221895adccfb3b692060251dd6a0183bc410 from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:48:36.903 INFO:tasks.cephadm:Discovered cachra url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T02:48:36.904 INFO:tasks.cephadm:Downloading cephadm from url: https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm 2024-09-21T02:48:36.904 DEBUG:teuthology.orchestra.run.smithi016:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:37.535 INFO:teuthology.orchestra.run.smithi016.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:48 /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:37.535 DEBUG:teuthology.orchestra.run.smithi018:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.170 INFO:teuthology.orchestra.run.smithi018.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:48 /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.171 DEBUG:teuthology.orchestra.run.smithi123:> curl --silent -L https://4.chacra.ceph.com/binaries/ceph/wip-pdonnell-testing-20240920.212106-debug/1843221895adccfb3b692060251dd6a0183bc410/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.719 INFO:teuthology.orchestra.run.smithi123.stdout:-rw-r--r--. 1 ubuntu ubuntu 804283 Sep 21 02:48 /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.719 DEBUG:teuthology.orchestra.run.smithi016:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.739 DEBUG:teuthology.orchestra.run.smithi018:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.758 DEBUG:teuthology.orchestra.run.smithi123:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-09-21T02:48:38.796 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 on all hosts... 2024-09-21T02:48:38.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:48:38.798 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:48:38.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 pull 2024-09-21T02:48:40.438 INFO:teuthology.orchestra.run.smithi018.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:48:40.448 INFO:teuthology.orchestra.run.smithi123.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:48:40.474 INFO:teuthology.orchestra.run.smithi016.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:49:10.321 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T02:49:10.322 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:49:10.322 INFO:teuthology.orchestra.run.smithi016.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:49:10.322 INFO:teuthology.orchestra.run.smithi016.stdout: "repo_digests": [ 2024-09-21T02:49:10.322 INFO:teuthology.orchestra.run.smithi016.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:49:10.322 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T02:49:10.323 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T02:49:51.188 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-21T02:49:51.188 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:49:51.188 INFO:teuthology.orchestra.run.smithi018.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:49:51.188 INFO:teuthology.orchestra.run.smithi018.stdout: "repo_digests": [ 2024-09-21T02:49:51.188 INFO:teuthology.orchestra.run.smithi018.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:49:51.189 INFO:teuthology.orchestra.run.smithi018.stdout: ] 2024-09-21T02:49:51.189 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev)", 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout: "image_id": "ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977", 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout: "repo_digests": [ 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262" 2024-09-21T02:49:51.555 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2024-09-21T02:49:51.556 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T02:49:51.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo mkdir -p /etc/ceph 2024-09-21T02:49:51.616 DEBUG:teuthology.orchestra.run.smithi018:> sudo mkdir -p /etc/ceph 2024-09-21T02:49:51.648 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph 2024-09-21T02:49:51.676 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 777 /etc/ceph 2024-09-21T02:49:51.705 DEBUG:teuthology.orchestra.run.smithi018:> sudo chmod 777 /etc/ceph 2024-09-21T02:49:51.733 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /etc/ceph 2024-09-21T02:49:51.758 INFO:tasks.cephadm:Writing seed config... 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-09-21T02:49:51.759 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] bdev async discard = True 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] bdev enable discard = True 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] bluestore allocator = bitmap 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] debug bluefs = 1/20 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] debug bluestore = 1/20 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] debug rocksdb = 4/10 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-09-21T02:49:51.760 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-09-21T02:49:51.761 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10 2024-09-21T02:49:51.761 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:49:51.761 DEBUG:teuthology.orchestra.run.smithi016:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-09-21T02:49:51.780 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = ffc7910c-77c3-11ef-baf5-efdc52797490 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bdev async discard = True bdev enable discard = True bluestore allocator = bitmap bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 1/20 debug bluestore = 1/20 debug ms = 1 debug osd = 20 debug rocksdb = 4/10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd_max_omap_entries_per_request = 10 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true 2024-09-21T02:49:51.780 DEBUG:teuthology.orchestra.run.smithi016:mon.a> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service 2024-09-21T02:49:51.823 DEBUG:teuthology.orchestra.run.smithi016:mgr.x> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x.service 2024-09-21T02:49:51.866 INFO:tasks.cephadm:Bootstrapping... 2024-09-21T02:49:51.866 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 -v bootstrap --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.16 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:49:52.085 INFO:teuthology.orchestra.run.smithi016.stdout:-------------------------------------------------------------------------------- 2024-09-21T02:49:52.086 INFO:teuthology.orchestra.run.smithi016.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', '-v', 'bootstrap', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.16', '--skip-admin-label'] 2024-09-21T02:49:52.086 INFO:teuthology.orchestra.run.smithi016.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-09-21T02:49:52.086 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying podman|docker is present... 2024-09-21T02:49:52.112 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-21T02:49:52.112 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying lvm2 is present... 2024-09-21T02:49:52.112 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying time synchronization is in place... 2024-09-21T02:49:52.120 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T02:49:52.120 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T02:49:52.127 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T02:49:52.127 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:49:52.134 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-21T02:49:52.141 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-21T02:49:52.141 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-21T02:49:52.141 INFO:teuthology.orchestra.run.smithi016.stdout:Repeating the final host check... 2024-09-21T02:49:52.165 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout 5.2.2 2024-09-21T02:49:52.166 INFO:teuthology.orchestra.run.smithi016.stdout:podman (/bin/podman) version 5.2.2 is present 2024-09-21T02:49:52.166 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl is present 2024-09-21T02:49:52.166 INFO:teuthology.orchestra.run.smithi016.stdout:lvcreate is present 2024-09-21T02:49:52.172 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-09-21T02:49:52.172 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-09-21T02:49:52.180 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-09-21T02:49:52.180 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:49:52.187 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout enabled 2024-09-21T02:49:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout active 2024-09-21T02:49:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:Unit chronyd.service is enabled and running 2024-09-21T02:49:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:Host looks OK 2024-09-21T02:49:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:Cluster fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:49:52.195 INFO:teuthology.orchestra.run.smithi016.stdout:Acquiring lock 140681630316912 on /run/cephadm/ffc7910c-77c3-11ef-baf5-efdc52797490.lock 2024-09-21T02:49:52.196 INFO:teuthology.orchestra.run.smithi016.stdout:Lock 140681630316912 acquired on /run/cephadm/ffc7910c-77c3-11ef-baf5-efdc52797490.lock 2024-09-21T02:49:52.196 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 3300 ... 2024-09-21T02:49:52.196 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying IP 172.21.15.16 port 6789 ... 2024-09-21T02:49:52.196 INFO:teuthology.orchestra.run.smithi016.stdout:Base mon IP(s) is [172.21.15.16:3300, 172.21.15.16:6789], mon addrv is [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T02:49:52.200 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via 172.21.15.254 dev ens1f0 proto dhcp src 172.21.15.16 metric 100 2024-09-21T02:49:52.200 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 172.21.0.0/20 dev ens1f0 proto kernel scope link src 172.21.15.16 metric 100 2024-09-21T02:49:52.202 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-09-21T02:49:52.202 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout fe80::/64 dev ens1f0 proto kernel metric 256 pref medium 2024-09-21T02:49:52.203 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev ens1f0 proto ra metric 1024 expires 1792sec hoplimit 64 pref medium 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout 4: ens1f0: mtu 1500 state UP qlen 1000 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-21T02:49:52.205 INFO:teuthology.orchestra.run.smithi016.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-09-21T02:49:52.206 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-21T02:49:52.206 INFO:teuthology.orchestra.run.smithi016.stdout:Mon IP `172.21.15.16` is in CIDR network `172.21.0.0/20` 2024-09-21T02:49:52.206 INFO:teuthology.orchestra.run.smithi016.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-09-21T02:49:52.207 INFO:teuthology.orchestra.run.smithi016.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-09-21T02:49:52.207 INFO:teuthology.orchestra.run.smithi016.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stdout ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410... 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Getting image source signatures 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:8425bc7408d91b180e7fdc62f23e7ae6603757f4f36e9969a4eaf9fd9416b764 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying blob sha256:e8b54c863393b7a8216a71737771b73b16a8e43ec7acf927c7faa175c255e551 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Copying config sha256:ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 2024-09-21T02:49:52.695 INFO:teuthology.orchestra.run.smithi016.stdout:/bin/podman: stderr Writing manifest to image destination 2024-09-21T02:49:53.591 INFO:teuthology.orchestra.run.smithi016.stdout:ceph: stdout ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T02:49:53.592 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph version: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev) 2024-09-21T02:49:53.592 INFO:teuthology.orchestra.run.smithi016.stdout:Extracting ceph user uid/gid from container image... 2024-09-21T02:49:54.616 INFO:teuthology.orchestra.run.smithi016.stdout:stat: stdout 167 167 2024-09-21T02:49:54.616 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial keys... 2024-09-21T02:49:55.671 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBSNO5m+9XoOBAA6X0nyyhiRhViwnOjalFYGg== 2024-09-21T02:49:56.577 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBUNO5mqGQgABAAP36Z0/RFXeCqtVDXwpgdUw== 2024-09-21T02:49:57.483 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph-authtool: stdout AQBUNO5mP6aZNRAAYC1zM0nXmaunD21OPeGh5A== 2024-09-21T02:49:57.483 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial monmap... 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:monmaptool for a [v2:172.21.15.16:3300,v1:172.21.15.16:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:setting min_mon_release = quincy 2024-09-21T02:49:58.405 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: set fsid to ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:49:58.406 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-09-21T02:49:58.406 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:49:58.406 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mon... 2024-09-21T02:49:59.413 INFO:teuthology.orchestra.run.smithi016.stdout:create mon.a on 2024-09-21T02:49:59.621 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-09-21T02:49:59.767 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-09-21T02:49:59.967 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target. 2024-09-21T02:49:59.967 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target → /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target. 2024-09-21T02:50:00.227 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a 2024-09-21T02:50:00.228 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service: Unit ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service not loaded. 2024-09-21T02:50:00.409 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target.wants/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service → /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@.service. 2024-09-21T02:50:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:50:00.927 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T02:50:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:50:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:50:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T02:50:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T02:50:00.934 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon to start... 2024-09-21T02:50:00.935 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mon... 2024-09-21T02:50:01.964 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout cluster: 2024-09-21T02:50:01.965 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout id: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout services: 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.547778s) 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout data: 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout pgs: 2024-09-21T02:50:01.966 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:01.967 INFO:teuthology.orchestra.run.smithi016.stdout:mon is available 2024-09-21T02:50:01.967 INFO:teuthology.orchestra.run.smithi016.stdout:Assimilating anything we can from ceph.conf... 2024-09-21T02:50:03.076 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:02 smithi016 ceph-mon[30884]: from='client.? 172.21.15.16:0/1861126768' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-09-21T02:50:03.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:03.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T02:50:03.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:03.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T02:50:03.080 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T02:50:03.081 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T02:50:03.082 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T02:50:03.082 INFO:teuthology.orchestra.run.smithi016.stdout:Generating new minimal ceph.conf... 2024-09-21T02:50:04.149 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:03 smithi016 ceph-mon[30884]: from='client.? 172.21.15.16:0/3446905298' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:50:04.152 INFO:teuthology.orchestra.run.smithi016.stdout:Restarting the monitor... 2024-09-21T02:50:04.445 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:04 smithi016 systemd[1]: Stopping Ceph mon.a for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:50:04.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a[30880]: 2024-09-21T02:50:04.443+0000 7f107c34f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T02:50:04.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a[30880]: 2024-09-21T02:50:04.443+0000 7f107c34f640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-09-21T02:50:04.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:04 smithi016 podman[31128]: 2024-09-21 02:50:04.668817099 +0000 UTC m=+0.342785651 container died 1a949b01058c5a4364f6c475634dbd70875af3140aa2e70950e5ef0221f09dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T02:50:05.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31128]: 2024-09-21 02:50:05.034696146 +0000 UTC m=+0.708664697 container cleanup 1a949b01058c5a4364f6c475634dbd70875af3140aa2e70950e5ef0221f09dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T02:50:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 bash[31128]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a 2024-09-21T02:50:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31140]: 2024-09-21 02:50:05.31058047 +0000 UTC m=+0.637298229 container remove 1a949b01058c5a4364f6c475634dbd70875af3140aa2e70950e5ef0221f09dbc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:50:05.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service: Deactivated successfully. 2024-09-21T02:50:05.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 systemd[1]: Stopped Ceph mon.a for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:50:05.644 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 systemd[1]: Starting Ceph mon.a for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:50:05.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31234]: 2024-09-21 02:50:05.545613828 +0000 UTC m=+0.016544084 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T02:50:05.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31234]: 2024-09-21 02:50:05.64471624 +0000 UTC m=+0.115646488 container create 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T02:50:05.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31234]: 2024-09-21 02:50:05.863682836 +0000 UTC m=+0.334613092 container init 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T02:50:05.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 podman[31234]: 2024-09-21 02:50:05.866961048 +0000 UTC m=+0.337891302 container start 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:50:05.971 INFO:teuthology.orchestra.run.smithi016.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-09-21T02:50:06.200 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T02:50:06.200 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T02:50:06.200 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: pidfile_write: ignore empty --pid-file 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: load: jerasure load: lrc 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: RocksDB version: 7.9.2 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Git sha 0 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: DB SUMMARY 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: DB Session ID: BO578XI61LK5VQB6VTZZ 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: CURRENT file: CURRENT 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 94602 ; 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.error_if_exists: 0 2024-09-21T02:50:06.201 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.create_if_missing: 0 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.paranoid_checks: 1 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.env: 0x55d211fe7d60 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.info_log: 0x55d216989360 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.statistics: (nil) 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.use_fsync: 0 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_log_file_size: 0 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T02:50:06.202 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_fallocate: 1 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.use_direct_reads: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.db_log_dir: 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.wal_dir: 2024-09-21T02:50:06.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T02:50:06.204 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.write_buffer_manager: 0x55d21698dd60 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.unordered_write: 0 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T02:50:06.205 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.row_cache: None 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.wal_filter: None 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.two_write_queues: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.wal_compression: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.atomic_flush: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.log_readahead_size: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T02:50:06.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_background_jobs: 2 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_background_compactions: -1 2024-09-21T02:50:06.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_subcompactions: 1 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T02:50:06.208 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_open_files: -1 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_background_flushes: -1 2024-09-21T02:50:06.209 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Compression algorithms supported: 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kZSTD supported: 0 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kXpressCompression supported: 0 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kBZip2Compression supported: 0 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kLZ4Compression supported: 1 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kZlibCompression supported: 1 2024-09-21T02:50:06.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: kSnappyCompression supported: 1 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.merge_operator: 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_filter: None 2024-09-21T02:50:06.211 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x55d216988fa0) 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks: 1 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T02:50:06.212 INFO:journalctl@ceph.mon.a.smithi016.stdout: pin_top_level_index_and_filter: 1 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_type: 0 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_index_type: 0 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_shortening: 1 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: checksum: 4 2024-09-21T02:50:06.213 INFO:journalctl@ceph.mon.a.smithi016.stdout: no_block_cache: 0 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache: 0x55d21294d610 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_name: BinnedLRUCache 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_options: 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: capacity : 536870912 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_shard_bits : 4 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: strict_capacity_limit : 0 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: high_pri_pool_ratio: 0.000 2024-09-21T02:50:06.214 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_cache_compressed: (nil) 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: persistent_cache: (nil) 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size: 4096 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_size_deviation: 10 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_restart_interval: 16 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: index_block_restart_interval: 1 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: metadata_block_size: 4096 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: partition_filters: 0 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: use_delta_encoding: 1 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: filter_policy: bloomfilter 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: whole_key_filtering: 1 2024-09-21T02:50:06.215 INFO:journalctl@ceph.mon.a.smithi016.stdout: verify_compression: 0 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: read_amp_bytes_per_bit: 0 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: format_version: 5 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_index_compression: 1 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: block_align: 0 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: max_auto_readahead_size: 262144 2024-09-21T02:50:06.217 INFO:journalctl@ceph.mon.a.smithi016.stdout: prepopulate_block_cache: 0 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout: initial_auto_readahead_size: 8192 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression: NoCompression 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.num_levels: 7 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T02:50:06.218 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T02:50:06.219 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T02:50:06.220 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T02:50:06.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T02:50:06.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.inplace_update_support: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.bloom_locality: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.max_successive_merges: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T02:50:06.223 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.ttl: 2592000 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enable_blob_files: false 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.min_blob_size: 0 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T02:50:06.224 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T02:50:06.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T02:50:06.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T02:50:06.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-09-21T02:50:06.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-09-21T02:50:06.225 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 95602a66-ef0f-46bb-9449-8031d07034dc 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887005902525, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887005903733, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 90828, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 283, "table_properties": {"data_size": 88869, "index_size": 279, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 645, "raw_key_size": 11567, "raw_average_key_size": 47, "raw_value_size": 82332, "raw_average_value_size": 340, "num_data_blocks": 12, "num_entries": 242, "num_filter_entries": 242, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726887005, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "95602a66-ef0f-46bb-9449-8031d07034dc", "db_session_id": "BO578XI61LK5VQB6VTZZ", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887005903840, "job": 1, "event": "recovery_finished"} 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x55d2169aca80 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: DB pointer 0x55d217752000 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** DB Stats ** 2024-09-21T02:50:06.226 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: L0 2/0 90.56 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Sum 2/0 90.56 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 120.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.227 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** Compaction Stats [default] ** 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 120.0 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Cumulative compaction: 0.00 GB write, 25.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Interval compaction: 0.00 GB write, 25.19 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T02:50:06.228 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache BinnedLRUCache@0x55d21294d610#2 capacity: 512.00 MB usage: 1.31 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,0.84 KB,0.000160933%) IndexBlock(2,0.47 KB,8.9407e-05%) Misc(1,0.00 KB,0%) 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: starting mon.a rank 0 at public addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] at bind addrs [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???) e1 preinit fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).mds e1 new map 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).mds e1 print_map 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: e1 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: btime 2024-09-21T02:50:00:888218+0000 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: legacy client fscid: -1 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout: No filesystems configured 2024-09-21T02:50:06.229 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mon.a is new leader, mons a in quorum (ranks 0) 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: monmap epoch 1 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: last_changed 2024-09-21T02:49:57.809427+0000 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: min_mon_release 19 (squid) 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: election_strategy: 1 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: fsmap 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: osdmap e1: 0 total, 0 up, 0 in 2024-09-21T02:50:06.230 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 ceph-mon[31253]: mgrmap e1: no daemons active 2024-09-21T02:50:06.231 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 bash[31234]: 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a 2024-09-21T02:50:06.231 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:05 smithi016 systemd[1]: Started Ceph mon.a for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:50:07.007 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote config to /etc/ceph/ceph.conf 2024-09-21T02:50:07.020 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:50:07.020 INFO:teuthology.orchestra.run.smithi016.stdout:Creating mgr... 2024-09-21T02:50:07.021 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:9283 ... 2024-09-21T02:50:07.021 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying port 0.0.0.0:8765 ... 2024-09-21T02:50:07.252 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x 2024-09-21T02:50:07.253 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Failed to reset failed state of unit ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x.service: Unit ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x.service not loaded. 2024-09-21T02:50:07.411 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490.target.wants/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x.service → /etc/systemd/system/ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@.service. 2024-09-21T02:50:07.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:07 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3546004542' entity='client.admin' 2024-09-21T02:50:07.933 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:07 smithi016 podman[31466]: 2024-09-21 02:50:07.86091522 +0000 UTC m=+0.325310467 container start 136364bf88ff729dbcc607c04c160d9a3afadc063d0597453945c6f03a1b4c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T02:50:07.934 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:07 smithi016 bash[31466]: 136364bf88ff729dbcc607c04c160d9a3afadc063d0597453945c6f03a1b4c46 2024-09-21T02:50:07.948 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:50:07.948 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T02:50:07.956 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:50:07.956 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:50:07.956 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T02:50:07.956 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to enable service . firewalld.service is not available 2024-09-21T02:50:07.964 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:50:07.964 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T02:50:07.972 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:50:07.972 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:50:07.972 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T02:50:07.973 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-09-21T02:50:07.973 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr to start... 2024-09-21T02:50:07.973 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr... 2024-09-21T02:50:08.194 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:07 smithi016 systemd[1]: Started Ceph mgr.x for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:50:08.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:08 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4240005056' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:50:09.114 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ffc7910c-77c3-11ef-baf5-efdc52797490", 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:50:09.115 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.116 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:50:00:888218+0000", 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:50:09.117 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:50:00.889020+0000", 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:09.118 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (1/15)... 2024-09-21T02:50:11.879 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:11 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1122145495' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ffc7910c-77c3-11ef-baf5-efdc52797490", 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:50:12.235 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 5, 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.236 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:50:12.237 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:50:12.240 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:50:00:888218+0000", 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": false, 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:12.241 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:50:00.889020+0000", 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:12.242 INFO:teuthology.orchestra.run.smithi016.stdout:mgr not available, waiting (2/15)... 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: Activating manager daemon x 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: mgrmap e2: x(active, starting, since 0.00366393s) 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: Manager daemon x is now available 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' 2024-09-21T02:50:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' 2024-09-21T02:50:13.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:13 smithi016 ceph-mon[31253]: from='mgr.14100 172.21.15.16:0/700450769' entity='mgr.x' 2024-09-21T02:50:15.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:15 smithi016 ceph-mon[31253]: mgrmap e3: x(active, since 1.00859s) 2024-09-21T02:50:15.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3271965823' entity='client.admin' cmd={"prefix": "status", "format": "json-pretty"} : dispatch 2024-09-21T02:50:15.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:15.566 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsid": "ffc7910c-77c3-11ef-baf5-efdc52797490", 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "health": { 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 0 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "a" 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:15.567 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "monmap": { 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-09-21T02:50:15.568 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-09-21T02:50:15.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.571 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "btime": "2024-09-21T02:50:00:888218+0000", 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-09-21T02:50:15.572 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modules": [ 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "iostat", 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "nfs", 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "restful" 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ], 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "modified": "2024-09-21T02:50:00.889020+0000", 2024-09-21T02:50:15.573 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "services": {} 2024-09-21T02:50:15.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout }, 2024-09-21T02:50:15.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-09-21T02:50:15.574 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:15.574 INFO:teuthology.orchestra.run.smithi016.stdout:mgr is available 2024-09-21T02:50:16.237 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:16 smithi016 ceph-mon[31253]: mgrmap e4: x(active, since 2s) 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [global] 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout fsid = ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.16:3300,v1:172.21.15.16:6789] 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-09-21T02:50:16.797 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [mgr] 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout [osd] 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout bdev_async_discard = True 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-09-21T02:50:16.798 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling cephadm module... 2024-09-21T02:50:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:17 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1879285983' entity='client.admin' cmd={"prefix": "config assimilate-conf"} : dispatch 2024-09-21T02:50:18.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:18 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/852708633' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "cephadm"} : dispatch 2024-09-21T02:50:18.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setuser ceph since I am not root 2024-09-21T02:50:18.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setgroup ceph since I am not root 2024-09-21T02:50:19.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:19 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/852708633' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-09-21T02:50:19.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:19 smithi016 ceph-mon[31253]: mgrmap e5: x(active, since 5s) 2024-09-21T02:50:20.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:20.075 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-21T02:50:20.076 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 5... 2024-09-21T02:50:20.474 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/571791571' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: Active manager daemon x restarted 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: Activating manager daemon x 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: osdmap e2: 0 total, 0 up, 0 in 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: mgrmap e6: x(active, starting, since 0.00408636s) 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:50:23.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:23 smithi016 ceph-mon[31253]: Manager daemon x is now available 2024-09-21T02:50:25.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:25.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-09-21T02:50:25.047 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T02:50:25.048 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:25.048 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 5 is available 2024-09-21T02:50:25.048 INFO:teuthology.orchestra.run.smithi016.stdout:Verifying orchestrator module is enabled... 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: Found migration_current of "None". Setting to last migration. 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T02:50:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: mgrmap e7: x(active, since 1.00718s) 2024-09-21T02:50:25.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-mon[31253]: from='client.14122 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T02:50:26.266 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:50:25.773+0000 7f88488e9640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:50:25] ENGINE Error in HTTPServer.serve 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Traceback (most recent call last): 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._connections.run(self.expiration_interval) 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._run(expiration_interval) 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s = self.context.wrap_socket( 2024-09-21T02:50:26.267 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: return self.sslsocket_class._create( 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self.do_handshake() 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._sslobj.do_handshake() 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:50:26.268 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:50:26.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:25] ENGINE Bus STARTING 2024-09-21T02:50:26.642 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:25] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3348071074' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "orchestrator"} : dispatch 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:25] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:25] ENGINE Bus STARTED 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:26 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:25] ENGINE Error in HTTPServer.serve 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:50:26.643 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-21T02:50:26.644 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:50:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr module 'orchestrator' is already enabled (always-on) 2024-09-21T02:50:26.955 INFO:teuthology.orchestra.run.smithi016.stdout:Setting orchestrator backend to cephadm... 2024-09-21T02:50:27.620 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:27 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3348071074' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "orchestrator"}]': finished 2024-09-21T02:50:27.621 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:27 smithi016 ceph-mon[31253]: mgrmap e8: x(active, since 2s) 2024-09-21T02:50:28.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:28 smithi016 ceph-mon[31253]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:28.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:28 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:28.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:28 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:29.508 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout value unchanged 2024-09-21T02:50:29.508 INFO:teuthology.orchestra.run.smithi016.stdout:Generating ssh key... 2024-09-21T02:50:29.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:29 smithi016 ceph-mon[31253]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:30.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-mon[31253]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-mon[31253]: Generating ssh key... 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Generating public/private rsa key pair. 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Your identification has been saved in /tmp/tmput4jpfec/key 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Your public key has been saved in /tmp/tmput4jpfec/key.pub 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: The key fingerprint is: 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: SHA256:53bVwYO20x/MhJ7uVU2NwikxyZEoAqux5Pi3P9MzCyQ ceph-ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: The key's randomart image is: 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: +---[RSA 3072]----+ 2024-09-21T02:50:30.766 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | .. o++ | 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | .. . . ++ .+..| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: |... . . . ++.=o| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: |++ .o.Bo+| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: |+. E . S . =.=+| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | . o o ... +| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | . . .. o .. ..| 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | . .o.+. .. . | 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: | ...o.+ . | 2024-09-21T02:50:30.767 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: +----[SHA256]-----+ 2024-09-21T02:50:32.348 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:50:32.348 INFO:teuthology.orchestra.run.smithi016.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-09-21T02:50:32.348 INFO:teuthology.orchestra.run.smithi016.stdout:Adding key to root@localhost authorized_keys... 2024-09-21T02:50:32.351 INFO:teuthology.orchestra.run.smithi016.stdout:Adding host smithi016... 2024-09-21T02:50:32.581 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:32 smithi016 ceph-mon[31253]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:33.426 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:33 smithi016 ceph-mon[31253]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi016", "addr": "172.21.15.16", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:34.589 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:34 smithi016 ceph-mon[31253]: Deploying cephadm binary to smithi016 2024-09-21T02:50:37.456 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Added host 'smithi016' with addr '172.21.15.16' 2024-09-21T02:50:37.456 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mon service... 2024-09-21T02:50:38.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:37 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:38.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:37 smithi016 ceph-mon[31253]: Added host smithi016 2024-09-21T02:50:38.132 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:37 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:39.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:39 smithi016 ceph-mon[31253]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:39.457 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:39 smithi016 ceph-mon[31253]: Saving service mon spec with placement count:5 2024-09-21T02:50:39.458 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:39 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:39.460 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-09-21T02:50:39.460 INFO:teuthology.orchestra.run.smithi016.stdout:Deploying unmanaged mgr service... 2024-09-21T02:50:40.861 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-09-21T02:50:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:41 smithi016 ceph-mon[31253]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:41 smithi016 ceph-mon[31253]: Saving service mgr spec with placement count:2 2024-09-21T02:50:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:41 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:41 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:41 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:42.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:42 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2564554689' entity='client.admin' 2024-09-21T02:50:43.518 INFO:teuthology.orchestra.run.smithi016.stdout:Enabling the dashboard module... 2024-09-21T02:50:44.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:43 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2986073719' entity='client.admin' 2024-09-21T02:50:45.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:44 smithi016 ceph-mon[31253]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:50:45.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:44 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2411358117' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "dashboard"} : dispatch 2024-09-21T02:50:45.110 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:44 smithi016 ceph-mon[31253]: from='mgr.14118 172.21.15.16:0/422582616' entity='mgr.x' 2024-09-21T02:50:45.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setuser ceph since I am not root 2024-09-21T02:50:45.111 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setgroup ceph since I am not root 2024-09-21T02:50:45.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:45 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2411358117' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-09-21T02:50:45.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:45 smithi016 ceph-mon[31253]: mgrmap e9: x(active, since 21s) 2024-09-21T02:50:46.913 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:46 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3855488676' entity='client.admin' cmd={"prefix": "mgr stat"} : dispatch 2024-09-21T02:50:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-09-21T02:50:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "available": true, 2024-09-21T02:50:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-09-21T02:50:46.916 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-09-21T02:50:46.917 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:46.917 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for the mgr to restart... 2024-09-21T02:50:46.917 INFO:teuthology.orchestra.run.smithi016.stdout:Waiting for mgr epoch 9... 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: Active manager daemon x restarted 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: Activating manager daemon x 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: osdmap e3: 0 total, 0 up, 0 in 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: mgrmap e10: x(active, starting, since 0.00378858s) 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:50:50.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:50:50.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:50:50.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:50 smithi016 ceph-mon[31253]: Manager daemon x is now available 2024-09-21T02:50:51.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:51 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:50:51.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:51 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:50:51.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:51 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:50:51.511 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:51 smithi016 ceph-mon[31253]: mgrmap e11: x(active, since 1.00651s) 2024-09-21T02:50:51.805 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout { 2024-09-21T02:50:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-09-21T02:50:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout "initialized": true 2024-09-21T02:50:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout } 2024-09-21T02:50:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:mgr epoch 9 is available 2024-09-21T02:50:51.806 INFO:teuthology.orchestra.run.smithi016.stdout:Using certmgr to generate dashboard self-signed certificate... 2024-09-21T02:50:52.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-mon[31253]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-09-21T02:50:52.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-mon[31253]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-09-21T02:50:52.701 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:50:52.576+0000 7f0c2ec33640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:50:52] ENGINE Error in HTTPServer.serve 2024-09-21T02:50:52.701 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Traceback (most recent call last): 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._connections.run(self.expiration_interval) 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._run(expiration_interval) 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s = self.context.wrap_socket( 2024-09-21T02:50:52.702 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: return self.sslsocket_class._create( 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self.do_handshake() 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._sslobj.do_handshake() 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:50:52.703 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:50:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:50:53.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: mgrmap e12: x(active, since 2s) 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:52] ENGINE Bus STARTING 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "orch certmgr generate-certificates", "module_name": "dashboard", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:52] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:52] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:52] ENGINE Bus STARTED 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: [21/Sep/2024:02:50:52] ENGINE Error in HTTPServer.serve 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:50:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:50:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:53 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:50:54.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:54 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:50:55.103 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"cert": "-----BEGIN CERTIFICATE-----\nMIIE+zCCAuOgAwIBAgIUEIlcaex8wPzG76KXa+PQW80hFccwDQYJKoZIhvcNAQEL\nBQAwFzEVMBMGA1UEAwwMY2VwaGFkbS1yb290MB4XDTI0MDkyMTAyNTA1M1oXDTM0\nMDkyMjAyNTA1M1owFzEVMBMGA1UEAwwMMTcyLjIxLjE1LjE2MIICIjANBgkqhkiG\n9w0BAQEFAAOCAg8AMIICCgKCAgEAnV0+HVPc2vOj5ZSnuo7q7ktbOzOyxUu8Zpno\n32dwKUbZ2WoPu2IBnao0mJemXZa/EMQrmAG3FJ1QYdDylytrV58kyi0HaPN4AQl3\nrReFFnplHRC6HRtrtBd4PtY0W6TBA2R3K0/0iL9NfxuZBdOA3WKTcPuAsVShzOkw\nL56zDCXmeEbp72ueFXY35bY6LqSC/4YGtM0y2+Cl/0X8bU2XrjJC5AWpSbBwGxL0\nwsi0fM/yXnB8gTaT+jNLiKZyHEnsy7xa/ZfT0MNFQV0By6YErWy2iuh7Ge/+mcQR\nzP4WQJEhJqU2JGmCzekPa3U5W+yAlt+FZEc7snTwNWf44TTFnh06kvJOuLHT/8Qc\ntfpp6eg7bN3AmhtdQ0z0XZmkqrzMia6cDOfEiyeNv4j+PcGq5p1ympa88F2FD+0I\nBYYpXBr2dPxX1wn15/jIrbl/4OyV+B+E13fNlavVXz8xs6TdNUKGLiaggfh2MKEw\nFvqazCpSfN9qzt404th0+vCRF0VMraeJr6eRwRmdlTdMiV/QRNUZJy8Rp/zFJY2n\nzqx8p4w9JAZfzf/OZZo76sMxR0hxjycvAHAilu/A0s02m8JUCl19qOF4ZuHaN/QG\nBWdWvVyEBskdJNJRY9uuW0IMjKZ+/9AKyHYApXDBOS5+DM6oX4Dih5KzQQPUiTPQ\nD5Oqp/0CAwEAAaM/MD0wLQYDVR0RBCYwJIIJc21pdGhpMDE2ghFkYXNoYm9hcmRf\nc2VydmVyc4cErBUPEDAMBgNVHRMBAf8EAjAAMA0GCSqGSIb3DQEBCwUAA4ICAQBQ\no81DeH5UOg97W0Kmg5vPaSJXYIYZQPOEglAErXhsTCXgSNOql3ZAETEsOjFGFjVU\nD+d47oqA+4cWmOPSOTcbNL1UvUsO9kE/9RtDxHV7Kb72ql4Ei6szzXgC/kXSh3Kl\n7AUaOoKZTHKKHh4p9dxUxelbu1THXOU0hekcYSSd1DA7zjOS/DY+Dr5EdD8bwmYJ\n0suKZuBebcLCb7xw0Q8y+wmm4AxpqnW93FxJiDeiulYQwmXYhMVS7mhNGCmkZPyq\n+Brcp9mRMdTGqndz8FFlKQzNUfnCKwBnBN3Si5ry3U35/ZCwaPN519pp0jEa28nG\nZo9BilmxgIGwrHAyobGFnrsctUUrPXIlkOo8VFyCXrBVE3hO+lZteHUb5Z36ZsUy\ngVGUcTIcgomWjOA1IvVks4xCdeLCClFlqVH780nQw/wKeL58ZnWUH1h0NYvOsuUE\ntBYgkW66hNJcayv7+cZfBdLNPP2wupEf2BAqqrPTmWnBSfmoDkdgs8/dlCOBw2mm\nEDKRwJfAC1H0LD98Lv5PYGkUXVPpjzShdRb6uEAj48nWOABGmYSk3u0u6Nm8dCpG\nrU/Fv5lWSuJDsoJuX6DnF50z9eL8QK77VO0EREfIktCGTuxxy1JYoG2nOJO7sivJ\n/WMWX9DjliingvpO5cr9jXbS8dPH3dUHSe03AnTCQA==\n-----END CERTIFICATE-----\n", "key": "-----BEGIN RSA PRIVATE KEY-----\nMIIJKQIBAAKCAgEAnV0+HVPc2vOj5ZSnuo7q7ktbOzOyxUu8Zpno32dwKUbZ2WoP\nu2IBnao0mJemXZa/EMQrmAG3FJ1QYdDylytrV58kyi0HaPN4AQl3rReFFnplHRC6\nHRtrtBd4PtY0W6TBA2R3K0/0iL9NfxuZBdOA3WKTcPuAsVShzOkwL56zDCXmeEbp\n72ueFXY35bY6LqSC/4YGtM0y2+Cl/0X8bU2XrjJC5AWpSbBwGxL0wsi0fM/yXnB8\ngTaT+jNLiKZyHEnsy7xa/ZfT0MNFQV0By6YErWy2iuh7Ge/+mcQRzP4WQJEhJqU2\nJGmCzekPa3U5W+yAlt+FZEc7snTwNWf44TTFnh06kvJOuLHT/8Qctfpp6eg7bN3A\nmhtdQ0z0XZmkqrzMia6cDOfEiyeNv4j+PcGq5p1ympa88F2FD+0IBYYpXBr2dPxX\n1wn15/jIrbl/4OyV+B+E13fNlavVXz8xs6TdNUKGLiaggfh2MKEwFvqazCpSfN9q\nzt404th0+vCRF0VMraeJr6eRwRmdlTdMiV/QRNUZJy8Rp/zFJY2nzqx8p4w9JAZf\nzf/OZZo76sMxR0hxjycvAHAilu/A0s02m8JUCl19qOF4ZuHaN/QGBWdWvVyEBskd\nJNJRY9uuW0IMjKZ+/9AKyHYApXDBOS5+DM6oX4Dih5KzQQPUiTPQD5Oqp/0CAwEA\nAQKCAgAayJyCXFw6Tf6m3DLiH2IgFeZcwLNcPkQ+SFLo90+RTvRUO2ebEpbI0Yfr\nMuCHjFlPzrvGvv/8u7vLMkef8C1wd+9Ik/pvZAUYVBaw+NBXn5RnKooq2xhtShNW\nypmVwYhyfd6EmjK6NalIiZd1dOa5h4Yl1EC2rftuB4CCkmXSGv9LmHJHpE7/QyRH\nbkkf2DNNr2+jxUszPlyygqRwrCS1auuUqNwD5UpnaUYMgyGpHrGXXljqmzH7g1CH\nXSEN7I0nrhLkl7tfDGVUtzJPiZ3BMMAvC1uscfspC8Sl2ddKvQjh7Xu2Hdk8F79e\noFCXczz9sOSgyTkBakLaLCyhlBxQRRqA+KJQRcFcRUrF7rARsg0UhQIvRy5saP5j\nKDXucdSaVfLcP2V9wHegyx8E5ML1BOwunCCD5iUp/zeWzbgJtH6xD9Zury4hMjqe\nTZp+JXTTF+vg1Uef55lkjNB4/2vsZRJtXzT/bPgi8xNfp8t8q7BULozHF90n012B\neKRZdft+WTFfXhUzlzGBTyiznBjmcRYpvkaLUlJ3jmsZlLo8htDoqLG9grk7GeI4\n8cALelduwYZNx8I4JVyruFCb+QVvBXPphBS9ypHEo+3n0rDkAU1TqujnMLQevtOb\nl5sx93I0Iq/VywRtqHdqd9/bzv+HnQWu4ec99PkOeu6WT4Z9oQKCAQEA1osGRxmt\n2Dfosln51qx/q1MUOJB0SG8+zoswE7NzJGrm9morAZ9zdT6Nla2XXrDLkzFVf5GO\njWyl+lClPKE0BUoPCi2NzHEVjr/syvr8+Np7gEv/qbgDliA0cESzOmLRkJYjY8Ut\nxySXo/6pkvEk8El6LLS6TyRyIcbrIpkDaoUGsn7dFS4xZHlB+JKDvh1BMu/0vZ1u\n8b8h0pwrubN4btzm2wdzZHprvflZxeEMQbvlGlxVp9wBaeY0n6GHiSwwEI6EmY8F\nRzAa2cU4u3/5E6BchnWPZTVsCym6PXHan8j4mqD3d7sejdulkmMm1shSPxCO5u2b\nPSAwha1O6YStjQKCAQEAu8W01CrGNDEbOEsZ1d6GCh+h0g0nP/w+o/ZahtLlaleC\nvKppziUQ8zc9i9ApcyYhJcpxjL3LHfP9BFE5N/eMSQb7aN1bHbbA2VModK4jVKn5\nelR4KssUuYVKD90FYCnoG32UDBgq00oj/p68FiDS6KFVzd+7YcQUfK09QSBu9cJU\n4eKW0lDLoijkafQUjdLy4UWl357llkqPBOVslTir0H5flL8/S+/nqHn+15VOVICC\nSh8760laDntWZqziJ228VHahR8GuFzQQR71eTPf4r/9snDeDw3EDBunx/lUNAn7p\nNBsY18F6Vbowvo68dE3slgehrPN+gM8M3SjXIWcwMQKCAQANpqbtmzquiVArGQKG\noE83TX/54WckvL9YUQ7sEfz/kjXD9BlsbwLfsO09WuBIzCMH6JmyFkblxMYC2WLK\nKs+ujnnKgf+FxKREgv3EJJg4FV7KWQuZdvXzdXUgt55X1rXS51q1HHS/ORrU+n07\ntlBNYb9+XLmGE6GXeJq5htbqzjAQ53WVxWyrIfQKpvfu/tmJLi96aSwIActbJUi6\n9ipxk2i1AxJakFeOrPEDYEBxZk07kih5F8yXOmiax6JiHgeo5giV5L3HO85ZkOz9\nMTVeBFQe0hgfrniDdVI+WqI4jIcU/6z/V06oZBNy7xAnlnN1Q85z7yZzizrLJq3g\nuep5AoIBAQCxVyBZ4awgTuCuYf+chfWCSi3C/Lk9LTbcFCebfkM+FDhdh3U2RLul\neV6fVZJQNorH7lMf4YCcYXttQGkR+rJ7piR7/dO4VIn+txNWX8DujToZ1v1HLVX9\ngW1ZXh1dnTaiABB6+7g2pacpKzHolEedfgJT1iJbS50tw1A+2irx4P/Z9u8K41Pv\naz+O8f8Xk0vghq8KbD/ZmcFJloTeyg87cp9DFPzq80GPVOjjIxg/pr9m0ej9ZuGv\nG72Wa2GMa4+bFg9sffe14m4d0zPk7a3qSwqA382KIIoDVaA/3IPU/5ewpobn/stT\ne+3di2FhVtdDOyHu/mlTrjkwBwpnedWBAoIBAQDCNDBndh+MwjG6GNDCV2YmtnDJ\nzCb7tnC+n+X766oPz2PWA/Q/o5AxLvNxeOnRCMoVPydcuVEDdJsm35yBdUmYGj7/\nWs8p5NpP3P36bAbUzqquTT8GWZXXpyG1i+GEvjr0lN9pXpk2nkD4XP+kdLw8UrzG\nJlw2i4RiG/ZF53f0vLDmeKpI5d17iiNjE4yqDnB610tKn0EcjJfwVfSWQJfIyqcN\ncWEDkk/AjYf/uGxkCJw0T4g88ci8We2rS7+IxTCag2hW5FNau7cihH8qt9Pb1EIX\nCholMDiF8nzn7u06wqIVt27MiohqQfaQ51DbZydKo0kmHevq+l1xnbJiDtYy\n-----END RSA PRIVATE KEY-----\n"} 2024-09-21T02:50:56.837 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate updated 2024-09-21T02:50:57.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:57 smithi016 ceph-mon[31253]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:57.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:57 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:50:58.102 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout SSL certificate key updated 2024-09-21T02:50:58.103 INFO:teuthology.orchestra.run.smithi016.stdout:Creating initial admin user... 2024-09-21T02:50:58.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:58 smithi016 ceph-mon[31253]: from='client.14168 -' entity='client.admin' cmd=[{"prefix": "dashboard set-ssl-certificate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:58.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:58 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:50:59.630 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:59 smithi016 ceph-mon[31253]: from='client.14170 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:50:59.631 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:50:59 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:50:59.633 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$FCunzkP206HYAXWv21xg9OYGO5QEVhlcBqt/y973ZNatdvDAxOFvG", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1726887059, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-09-21T02:50:59.633 INFO:teuthology.orchestra.run.smithi016.stdout:Fetching dashboard port number... 2024-09-21T02:51:00.845 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:00 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1957340172' entity='client.admin' cmd={"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} : dispatch 2024-09-21T02:51:00.847 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stdout 8443 2024-09-21T02:51:00.857 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-09-21T02:51:00.858 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout disabled 2024-09-21T02:51:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-09-21T02:51:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:systemctl: stdout inactive 2024-09-21T02:51:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:firewalld.service is not enabled 2024-09-21T02:51:00.866 INFO:teuthology.orchestra.run.smithi016.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout:Ceph Dashboard is now available at: 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout: URL: https://smithi016.front.sepia.ceph.com:8443/ 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout: User: admin 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout: Password: g5a3slrh27 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:00.869 INFO:teuthology.orchestra.run.smithi016.stdout:Saving cluster configuration to /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config directory 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout:Or, if you are only running a single cluster on this host: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout:Please consider enabling telemetry to help improve Ceph: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.289 INFO:teuthology.orchestra.run.smithi016.stdout: ceph telemetry on 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:For more information see: 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:02.290 INFO:teuthology.orchestra.run.smithi016.stdout:Bootstrap complete. 2024-09-21T02:51:02.338 INFO:tasks.cephadm:Fetching config... 2024-09-21T02:51:02.339 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:02.339 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-09-21T02:51:02.357 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-09-21T02:51:02.357 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:02.357 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-09-21T02:51:02.414 INFO:tasks.cephadm:Fetching mon keyring... 2024-09-21T02:51:02.414 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:02.414 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/keyring of=/dev/stdout 2024-09-21T02:51:02.482 INFO:tasks.cephadm:Fetching pub ssh key... 2024-09-21T02:51:02.482 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:02.482 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-09-21T02:51:02.538 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-09-21T02:51:02.539 DEBUG:teuthology.orchestra.run.smithi016:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:51:02.619 INFO:teuthology.orchestra.run.smithi016.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:02.631 DEBUG:teuthology.orchestra.run.smithi018:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:51:02.673 INFO:teuthology.orchestra.run.smithi018.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:02.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-09-21T02:51:02.729 INFO:teuthology.orchestra.run.smithi123.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDhXiKf5pRn3x6H6NBzfpwoViq6iwO/BDdlCvo3d8i+LpkUNZXV3fSRKvD3qPV9289E/rvt+wov0O/MQC4fjEhpcvJGUWiyFKGxSDF77ptjc+TrJJVVJFsQxLN6Y/xb82iMV+0QAE31iVgEKSTYSk8qbr8gQtUeB94M90tlD76Bbl++ycQzZZkeBdnhaDCm6khD/Q9dRLJEEk+75KhO41LCLlPjTfk8M1Mie4W7nll/vohSnq3QQrSwc9C6iRlnAkgFeuNIqkwpQGkdOTRC4Rpl6Wr/2a9h827psUrnoHztxlV7zJWSKQ2zTNH32JTHJ2R4z2OcrToPZJ/FY+DKP7OtJA4bt42N5lvR3ntMfo8jGngeIFHbtRY5lotFX+MnNoInRxjUP0Lcy1jRNzX4CqkGApcpIgaiBOupruMs4zFvFIcoSNtZeWL5QHNeW+ElrALZnK3xay7bN+7+M3UABT6C9ro6Z+6sSnmd+yjTHTyXl2I+DcQqBXzec4OyHRV4fIM= ceph-ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:02.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-09-21T02:51:02.998 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:03.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/93436974' entity='client.admin' 2024-09-21T02:51:03.954 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:03 smithi016 ceph-mon[31253]: mgrmap e13: x(active, since 12s) 2024-09-21T02:51:04.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3483786709' entity='client.admin' 2024-09-21T02:51:04.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:04.716 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:04.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:04.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:04.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:04.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:04 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:04.746 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-09-21T02:51:04.746 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-09-21T02:51:04.997 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:06.412 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi018 2024-09-21T02:51:06.412 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:51:06.412 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/etc/ceph/ceph.conf 2024-09-21T02:51:06.432 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:51:06.432 DEBUG:teuthology.orchestra.run.smithi018:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:06.490 INFO:tasks.cephadm:Adding host smithi018 to orchestrator... 2024-09-21T02:51:06.490 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch host add smithi018 2024-09-21T02:51:06.740 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:06.955 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:06 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:08.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:07 smithi016 ceph-mon[31253]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:08.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:07 smithi016 ceph-mon[31253]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:09.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:08 smithi016 ceph-mon[31253]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi018", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:10.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:09 smithi016 ceph-mon[31253]: Deploying cephadm binary to smithi018 2024-09-21T02:51:11.179 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi018' with addr '172.21.15.18' 2024-09-21T02:51:11.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:11 smithi016 ceph-mon[31253]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:11.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:11 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:11.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:11 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:11.925 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:11 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:11.961 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T02:51:12.219 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:12.867 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:12.867 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}] 2024-09-21T02:51:13.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:12 smithi016 ceph-mon[31253]: Added host smithi018 2024-09-21T02:51:13.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:12 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:13.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:12 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:13.527 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi123 2024-09-21T02:51:13.527 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:51:13.527 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.conf 2024-09-21T02:51:13.546 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:51:13.546 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:13.604 INFO:tasks.cephadm:Adding host smithi123 to orchestrator... 2024-09-21T02:51:13.604 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch host add smithi123 2024-09-21T02:51:13.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:13 smithi016 ceph-mon[31253]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:13.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:13 smithi016 ceph-mon[31253]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:16.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:15 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:16 smithi016 ceph-mon[31253]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:16 smithi016 ceph-mon[31253]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:17.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:16 smithi016 ceph-mon[31253]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:17.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:16 smithi016 ceph-mon[31253]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:17.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:16 smithi016 ceph-mon[31253]: Deploying cephadm binary to smithi123 2024-09-21T02:51:18.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:17 smithi016 ceph-mon[31253]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:18.256 INFO:teuthology.orchestra.run.smithi016.stdout:Added host 'smithi123' with addr '172.21.15.123' 2024-09-21T02:51:18.927 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch host ls --format=json 2024-09-21T02:51:19.179 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:19.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: Added host smithi123 2024-09-21T02:51:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:19.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:19 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:19.822 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:51:19.822 INFO:teuthology.orchestra.run.smithi016.stdout:[{"addr": "172.21.15.16", "hostname": "smithi016", "labels": [], "status": ""}, {"addr": "172.21.15.18", "hostname": "smithi018", "labels": [], "status": ""}, {"addr": "172.21.15.123", "hostname": "smithi123", "labels": [], "status": ""}] 2024-09-21T02:51:20.442 INFO:tasks.cephadm:Setting crush tunables to default 2024-09-21T02:51:20.442 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd crush tunables default 2024-09-21T02:51:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:20.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:20 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:20.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:20 smithi016 ceph-mon[31253]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:21.577 INFO:teuthology.orchestra.run.smithi016.stderr:adjusted tunables profile to default 2024-09-21T02:51:22.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:21 smithi016 ceph-mon[31253]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:22.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:21 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2261159199' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T02:51:22.347 INFO:tasks.cephadm:Adding mon.a on smithi016 2024-09-21T02:51:22.347 INFO:tasks.cephadm:Adding mon.b on smithi018 2024-09-21T02:51:22.347 INFO:tasks.cephadm:Adding mon.c on smithi123 2024-09-21T02:51:22.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch apply mon '3;smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c' 2024-09-21T02:51:22.598 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /etc/ceph/ceph.conf 2024-09-21T02:51:22.641 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2261159199' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:22 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:23.309 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mon update... 2024-09-21T02:51:23.912 DEBUG:teuthology.orchestra.run.smithi018:mon.b> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.b.service 2024-09-21T02:51:23.915 DEBUG:teuthology.orchestra.run.smithi123:mon.c> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.c.service 2024-09-21T02:51:23.917 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:51:23.917 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:24.195 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:24.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:24.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:24.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:24.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:24.196 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:23 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:24.602 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:25.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:24 smithi016 ceph-mon[31253]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:25.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:24 smithi016 ceph-mon[31253]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c;count:3 2024-09-21T02:51:25.198 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:24 smithi016 ceph-mon[31253]: Deploying daemon mon.c on smithi123 2024-09-21T02:51:25.266 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:51:25.266 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","modified":"2024-09-21T02:49:57.809427Z","created":"2024-09-21T02:49:57.809427Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T02:51:25.266 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-09-21T02:51:26.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:25 smithi016 ceph-mon[31253]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:26.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:25 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/1278404710' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:27.096 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:51:27.096 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:51:27.478 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:51:28.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:27 smithi016 ceph-mon[31253]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.321 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 systemd[1]: Starting Ceph mon.c for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:51:28.399 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:51:28.399 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":1,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","modified":"2024-09-21T02:49:57.809427Z","created":"2024-09-21T02:49:57.809427Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-09-21T02:51:28.399 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 1 2024-09-21T02:51:28.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 podman[34612]: 2024-09-21 02:51:28.319561102 +0000 UTC m=+0.092737672 container create 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T02:51:28.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 podman[34612]: 2024-09-21 02:51:28.242729658 +0000 UTC m=+0.015906232 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:51:28.682 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 podman[34612]: 2024-09-21 02:51:28.681314863 +0000 UTC m=+0.454491425 container init 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 podman[34612]: 2024-09-21 02:51:28.684657458 +0000 UTC m=+0.457834019 container start 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: set uid:gid to 167:167 (ceph:ceph) 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: ceph version 19.3.0-5120-g18432218 (1843221895adccfb3b692060251dd6a0183bc410) squid (dev), process ceph-mon, pid 2 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pidfile_write: ignore empty --pid-file 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: load: jerasure load: lrc 2024-09-21T02:51:28.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: RocksDB version: 7.9.2 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Git sha 0 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Compile date 2024-09-20 22:29:49 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: DB SUMMARY 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: DB Session ID: 1AWJ3D33U58HBNOG3EBO 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: CURRENT file: CURRENT 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: IDENTITY file: IDENTITY 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.error_if_exists: 0 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.create_if_missing: 0 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.paranoid_checks: 1 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.flush_verify_memtable_count: 1 2024-09-21T02:51:28.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.env: 0x56279fbfbd60 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.fs: PosixFileSystem 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.info_log: 0x5627a4595360 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_file_opening_threads: 16 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.statistics: (nil) 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.use_fsync: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_log_file_size: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.log_file_time_to_roll: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.keep_log_file_num: 1000 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.recycle_log_file_num: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_fallocate: 1 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_mmap_reads: 0 2024-09-21T02:51:28.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_mmap_writes: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.use_direct_reads: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.create_missing_column_families: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.db_log_dir: 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.wal_dir: 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.table_cache_numshardbits: 6 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.WAL_ttl_seconds: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.WAL_size_limit_MB: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.is_fd_close_on_exec: 1 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.advise_random_on_open: 1 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.db_write_buffer_size: 0 2024-09-21T02:51:28.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.write_buffer_manager: 0x5627a4599d60 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.use_adaptive_mutex: 0 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.rate_limiter: (nil) 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.wal_recovery_mode: 2 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enable_thread_tracking: 0 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enable_pipelined_write: 0 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.unordered_write: 0 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.row_cache: None 2024-09-21T02:51:28.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.wal_filter: None 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_ingest_behind: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.two_write_queues: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.manual_wal_flush: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.wal_compression: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.atomic_flush: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-09-21T02:51:28.976 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.persist_stats_to_disk: 0 2024-09-21T02:51:28.977 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.write_dbid_to_manifest: 0 2024-09-21T02:51:28.977 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.log_readahead_size: 0 2024-09-21T02:51:28.977 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.best_efforts_recovery: 0 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.allow_data_in_errors: 0 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.db_host_id: __hostname__ 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enforce_single_del_contracts: true 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_background_jobs: 2 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_background_compactions: -1 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_subcompactions: 1 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.delayed_write_rate : 16777216 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_total_wal_size: 0 2024-09-21T02:51:28.978 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.stats_dump_period_sec: 600 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.stats_persist_period_sec: 600 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_open_files: -1 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bytes_per_sync: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.wal_bytes_per_sync: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.strict_bytes_per_sync: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_readahead_size: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_background_flushes: -1 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Compression algorithms supported: 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kZSTD supported: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kXpressCompression supported: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kBZip2Compression supported: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-09-21T02:51:28.979 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kLZ4Compression supported: 1 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kZlibCompression supported: 1 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kLZ4HCCompression supported: 1 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: kSnappyCompression supported: 1 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Fast CRC32 supported: Supported on x86 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: DMutex implementation: pthread_mutex_t 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.merge_operator: 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_filter: None 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_filter_factory: None 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.sst_partitioner_factory: None 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.memtable_factory: SkipListFactory 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.table_factory: BlockBasedTable 2024-09-21T02:51:28.980 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x5627a4594fa0) 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks: 1 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: pin_top_level_index_and_filter: 1 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_type: 0 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_index_type: 0 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_shortening: 1 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: data_block_hash_table_util_ratio: 0.750000 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: checksum: 4 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: no_block_cache: 0 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache: 0x5627a0559610 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_name: BinnedLRUCache 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_options: 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: capacity : 536870912 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_shard_bits : 4 2024-09-21T02:51:28.981 INFO:journalctl@ceph.mon.c.smithi123.stdout: strict_capacity_limit : 0 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: high_pri_pool_ratio: 0.000 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_cache_compressed: (nil) 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: persistent_cache: (nil) 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size: 4096 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_size_deviation: 10 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_restart_interval: 16 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: index_block_restart_interval: 1 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: metadata_block_size: 4096 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: partition_filters: 0 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: use_delta_encoding: 1 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: filter_policy: bloomfilter 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: whole_key_filtering: 1 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: verify_compression: 0 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: read_amp_bytes_per_bit: 0 2024-09-21T02:51:28.982 INFO:journalctl@ceph.mon.c.smithi123.stdout: format_version: 5 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_index_compression: 1 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: block_align: 0 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: max_auto_readahead_size: 262144 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: prepopulate_block_cache: 0 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: initial_auto_readahead_size: 8192 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout: num_file_reads_for_auto_readahead: 2 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.write_buffer_size: 33554432 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_write_buffer_number: 2 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression: NoCompression 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression: Disabled 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.prefix_extractor: nullptr 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.num_levels: 7 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-09-21T02:51:28.983 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.window_bits: -14 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.level: 32767 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.strategy: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-09-21T02:51:28.984 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.enabled: false 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.target_file_size_base: 67108864 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.target_file_size_multiplier: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-09-21T02:51:28.985 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.arena_block_size: 1048576 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.disable_auto_compactions: 0 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-09-21T02:51:28.986 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.inplace_update_support: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.inplace_update_num_locks: 10000 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.memtable_huge_page_size: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.bloom_locality: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.max_successive_merges: 0 2024-09-21T02:51:28.987 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.optimize_filters_for_hits: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.paranoid_file_checks: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.force_consistency_checks: 1 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.report_bg_io_stats: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.ttl: 2592000 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.periodic_compaction_seconds: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enable_blob_files: false 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.min_blob_size: 0 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_file_size: 268435456 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_compression_type: NoCompression 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.enable_blob_garbage_collection: false 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-09-21T02:51:28.988 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.blob_file_starting_level: 0 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 55a551f6-d431-4992-9a40-be164faccad4 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887088721638, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887088722271, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1726887088, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "55a551f6-d431-4992-9a40-be164faccad4", "db_session_id": "1AWJ3D33U58HBNOG3EBO", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: EVENT_LOG_v1 {"time_micros": 1726887088722394, "job": 1, "event": "recovery_finished"} 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x5627a45b8a80 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: DB pointer 0x5627a535e000 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-09-21T02:51:28.989 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** DB Stats ** 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:51:28.990 INFO:journalctl@ceph.mon.c.smithi123.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** Compaction Stats [default] ** 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 2.6 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: AddFile(Keys): cumulative 0, interval 0 2024-09-21T02:51:28.991 INFO:journalctl@ceph.mon.c.smithi123.stdout: Cumulative compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: Interval compaction: 0.00 GB write, 0.41 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache BinnedLRUCache@0x5627a0559610#2 capacity: 512.00 MB usage: 0.23 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.12 KB,2.23517e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: ** File Read Latency Histogram By Level [default] ** 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: using public_addr v2:172.21.15.123:0/0 -> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: starting mon.c rank -1 at public addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] at bind addrs [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(???) e0 preinit fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).mds e1 new map 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).mds e1 print_map 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: e1 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: btime 2024-09-21T02:50:00:888218+0000 2024-09-21T02:51:28.992 INFO:journalctl@ceph.mon.c.smithi123.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: legacy client fscid: -1 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout: No filesystems configured 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:51:28.993 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Added host smithi018 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi123", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:28.994 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Deploying cephadm binary to smithi123 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Added host smithi123 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.995 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.14186 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2261159199' entity='client.admin' cmd={"prefix": "osd crush tunables", "profile": "default"} : dispatch 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2261159199' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:28.996 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.14190 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:51:28.997 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Saving service mon spec with placement smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c;count:3 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: Deploying daemon mon.c on smithi123 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/1278404710' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 ceph-mon[34645]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 bash[34612]: 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 2024-09-21T02:51:28.998 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:28 smithi123 systemd[1]: Started Ceph mon.c for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:51:30.171 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:51:30.172 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:51:30.440 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:51:34.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:34.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: mon.a calling monitor election 2024-09-21T02:51:34.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.081 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: mon.c calling monitor election 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: monmap epoch 2 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: last_changed 2024-09-21T02:51:28.750918+0000 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: min_mon_release 19 (squid) 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: election_strategy: 1 2024-09-21T02:51:34.082 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: fsmap 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: mgrmap e13: x(active, since 43s) 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: overall HEALTH_OK 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:34.083 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:33 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:34.130 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:51:34.130 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","modified":"2024-09-21T02:51:28.750918Z","created":"2024-09-21T02:49:57.809427Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T02:51:34.130 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2024-09-21T02:51:34.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:34.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: mon.a calling monitor election 2024-09-21T02:51:34.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: mon.c calling monitor election 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: monmap epoch 2 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: last_changed 2024-09-21T02:51:28.750918+0000 2024-09-21T02:51:34.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: min_mon_release 19 (squid) 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: election_strategy: 1 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: fsmap 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: mgrmap e13: x(active, since 43s) 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: overall HEALTH_OK 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:34.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:33 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:35.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:34 smithi016 ceph-mon[31253]: Deploying daemon mon.b on smithi018 2024-09-21T02:51:35.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:34 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/481951738' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:35.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:34 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:34 smithi123 ceph-mon[34645]: Deploying daemon mon.b on smithi018 2024-09-21T02:51:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:34 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/481951738' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:34 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:35.744 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:51:35.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:51:35.998 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:51:36.118 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:35 smithi123 ceph-mon[34645]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:36.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:35 smithi016 ceph-mon[31253]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:36.692 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:51:36.692 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":2,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","modified":"2024-09-21T02:51:28.750918Z","created":"2024-09-21T02:49:57.809427Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-09-21T02:51:36.692 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 2 2024-09-21T02:51:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:36 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/2058003401' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:37.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:36 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/2058003401' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:37 smithi018 systemd[1]: Started Ceph mon.b for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:51:38.274 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-09-21T02:51:38.275 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mon dump -f json 2024-09-21T02:51:38.678 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:51:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: mon.a calling monitor election 2024-09-21T02:51:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: mon.c calling monitor election 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: mon.b calling monitor election 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: monmap epoch 3 2024-09-21T02:51:42.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: last_changed 2024-09-21T02:51:37.543279+0000 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: min_mon_release 19 (squid) 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: election_strategy: 1 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: 2: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.b 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: fsmap 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: mgrmap e13: x(active, since 52s) 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: overall HEALTH_OK 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.935 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:42 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: mon.a calling monitor election 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: mon.c calling monitor election 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: mon.b calling monitor election 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:51:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: monmap epoch 3 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: last_changed 2024-09-21T02:51:37.543279+0000 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: min_mon_release 19 (squid) 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: election_strategy: 1 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: 2: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.b 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: fsmap 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: mgrmap e13: x(active, since 52s) 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: overall HEALTH_OK 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:42.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:42 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: mon.a calling monitor election 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: mon.c calling monitor election 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: mon.b calling monitor election 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: monmap epoch 3 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: last_changed 2024-09-21T02:51:37.543279+0000 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: created 2024-09-21T02:49:57.809427+0000 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: min_mon_release 19 (squid) 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: election_strategy: 1 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: 0: [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] mon.a 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: 1: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.c 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: 2: [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] mon.b 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: fsmap 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: osdmap e4: 0 total, 0 up, 0 in 2024-09-21T02:51:43.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: mgrmap e13: x(active, since 52s) 2024-09-21T02:51:43.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: overall HEALTH_OK 2024-09-21T02:51:43.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:43.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:43.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:43.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:42 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:43.082 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:51:43.083 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","modified":"2024-09-21T02:51:37.543279Z","created":"2024-09-21T02:49:57.809427Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:3300","nonce":0},{"type":"v1","addr":"172.21.15.16:6789","nonce":0}]},"addr":"172.21.15.16:6789/0","public_addr":"172.21.15.16:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:3300","nonce":0},{"type":"v1","addr":"172.21.15.123:6789","nonce":0}]},"addr":"172.21.15.123:6789/0","public_addr":"172.21.15.123:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:3300","nonce":0},{"type":"v1","addr":"172.21.15.18:6789","nonce":0}]},"addr":"172.21.15.18:6789/0","public_addr":"172.21.15.18:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1,2]} 2024-09-21T02:51:43.083 INFO:teuthology.orchestra.run.smithi123.stderr:dumped monmap epoch 3 2024-09-21T02:51:43.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:43 smithi123 ceph-mon[34645]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:43.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:43 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:43.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:43 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/3704290756' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:43.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:43 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.855 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-09-21T02:51:43.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config generate-minimal-conf 2024-09-21T02:51:43.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:43 smithi016 ceph-mon[31253]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:43.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:43 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:43.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:43 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/3704290756' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:43.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:43 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:43.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:43 smithi018 ceph-mon[34587]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:43.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:43 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:43.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:43 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/3704290756' entity='client.admin' cmd={"prefix": "mon dump", "format": "json"} : dispatch 2024-09-21T02:51:43.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:43 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:44.121 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:44.946 INFO:teuthology.orchestra.run.smithi016.stdout:# minimal ceph.conf for ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:44.946 INFO:teuthology.orchestra.run.smithi016.stdout:[global] 2024-09-21T02:51:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: fsid = ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:51:44.946 INFO:teuthology.orchestra.run.smithi016.stdout: mon_host = [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:44 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:45.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:45.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:44 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:45.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:44 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:45.805 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-09-21T02:51:45.806 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:45.806 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:51:45.879 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:51:45.879 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:45.948 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:51:45.948 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:51:45.981 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:51:45.981 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: Reconfiguring daemon mon.a on smithi016 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4000264937' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:45 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.046 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.047 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.047 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:51:46.047 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: Reconfiguring daemon mon.a on smithi016 2024-09-21T02:51:46.047 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4000264937' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:46.047 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:45 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:46.050 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:51:46.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.conf 2024-09-21T02:51:46.078 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:46.078 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:46.078 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Reconfiguring mon.a (unknown last config time)... 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: Reconfiguring daemon mon.a on smithi016 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4000264937' entity='client.admin' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:46.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:45 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:46.083 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:51:46.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:46.154 INFO:tasks.cephadm:enabling module snap_schedule 2024-09-21T02:51:46.154 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- sudo ceph --cluster ceph mgr module enable snap_schedule 2024-09-21T02:51:46.845 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:48.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:48.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:51:48.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:51:48.017 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setuser ceph since I am not root 2024-09-21T02:51:48.017 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ignoring --setgroup ceph since I am not root 2024-09-21T02:51:48.097 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:48.098 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:51:48.099 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:47 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:51:48.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='mgr.14152 172.21.15.16:0/4243472811' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:51:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:47 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd={"prefix": "mgr module enable", "module": "snap_schedule"} : dispatch 2024-09-21T02:51:48.460 INFO:tasks.cephadm:Adding mgr.x on smithi016 2024-09-21T02:51:48.461 INFO:tasks.cephadm:Adding mgr.y on smithi018 2024-09-21T02:51:48.461 INFO:tasks.cephadm:Adding mgr.z on smithi123 2024-09-21T02:51:48.461 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch apply mgr '3;smithi016=x;smithi018=y;smithi123=z' 2024-09-21T02:51:48.714 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:51:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:48 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:51:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:48 smithi016 ceph-mon[31253]: mgrmap e14: x(active, since 57s) 2024-09-21T02:51:49.051 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:48 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:51:49.051 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:48 smithi123 ceph-mon[34645]: mgrmap e14: x(active, since 57s) 2024-09-21T02:51:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:48 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4029068996' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-09-21T02:51:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:48 smithi018 ceph-mon[34587]: mgrmap e14: x(active, since 57s) 2024-09-21T02:51:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: Active manager daemon x restarted 2024-09-21T02:51:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: Activating manager daemon x 2024-09-21T02:51:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:51:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: mgrmap e15: x(active, starting, since 0.00508487s) 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:51:53.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:53 smithi018 ceph-mon[34587]: Manager daemon x is now available 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: Active manager daemon x restarted 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: Activating manager daemon x 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: mgrmap e15: x(active, starting, since 0.00508487s) 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:51:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:51:53.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:51:53.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:51:53.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:53 smithi123 ceph-mon[34645]: Manager daemon x is now available 2024-09-21T02:51:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: Active manager daemon x restarted 2024-09-21T02:51:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: Activating manager daemon x 2024-09-21T02:51:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: osdmap e5: 0 total, 0 up, 0 in 2024-09-21T02:51:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: mgrmap e15: x(active, starting, since 0.00508487s) 2024-09-21T02:51:53.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "x", "id": "x"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata"} : dispatch 2024-09-21T02:51:53.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata"} : dispatch 2024-09-21T02:51:53.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:53 smithi016 ceph-mon[31253]: Manager daemon x is now available 2024-09-21T02:51:54.077 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mgr update... 2024-09-21T02:51:54.379 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:54.379 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:51:54.379 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:51:54.379 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:54 smithi123 ceph-mon[34645]: mgrmap e16: x(active, since 1.00929s) 2024-09-21T02:51:54.379 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:51:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:51:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:54 smithi016 ceph-mon[31253]: mgrmap e16: x(active, since 1.00929s) 2024-09-21T02:51:54.412 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:51:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"} : dispatch 2024-09-21T02:51:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"} : dispatch 2024-09-21T02:51:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:54 smithi018 ceph-mon[34587]: mgrmap e16: x(active, since 1.00929s) 2024-09-21T02:51:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:54.681 DEBUG:teuthology.orchestra.run.smithi018:mgr.y> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.y.service 2024-09-21T02:51:54.684 DEBUG:teuthology.orchestra.run.smithi123:mgr.z> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.z.service 2024-09-21T02:51:54.686 INFO:tasks.cephadm:setting config [client] client_mount_timeout = 600 2024-09-21T02:51:54.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set client client_mount_timeout 600 2024-09-21T02:51:54.688 INFO:tasks.cephadm:setting config [client] debug_client = 20 2024-09-21T02:51:54.689 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set client debug_client 20 2024-09-21T02:51:54.731 INFO:tasks.cephadm:setting config [client] debug_ms = 1 2024-09-21T02:51:54.731 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set client debug_ms 1 2024-09-21T02:51:54.774 INFO:tasks.cephadm:setting config [client] rados_mon_op_timeout = 900 2024-09-21T02:51:54.774 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set client rados_mon_op_timeout 900 2024-09-21T02:51:54.817 INFO:tasks.cephadm:setting config [client] rados_osd_op_timeout = 900 2024-09-21T02:51:54.817 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set client rados_osd_op_timeout 900 2024-09-21T02:51:54.860 INFO:tasks.cephadm:setting config [mds] debug_mds = 20 2024-09-21T02:51:54.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds 20 2024-09-21T02:51:54.904 INFO:tasks.cephadm:setting config [mds] debug_mds_balancer = 20 2024-09-21T02:51:54.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds debug_mds_balancer 20 2024-09-21T02:51:54.956 INFO:tasks.cephadm:setting config [mds] debug_ms = 2 2024-09-21T02:51:54.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds debug_ms 2 2024-09-21T02:51:55.004 INFO:tasks.cephadm:setting config [mds] mds_bal_fragment_size_max = 10000 2024-09-21T02:51:55.004 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_fragment_size_max 10000 2024-09-21T02:51:55.051 INFO:tasks.cephadm:setting config [mds] mds_bal_merge_size = 5 2024-09-21T02:51:55.052 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_merge_size 5 2024-09-21T02:51:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:55.074 INFO:tasks.cephadm:setting config [mds] mds_bal_split_bits = 3 2024-09-21T02:51:55.074 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_bits 3 2024-09-21T02:51:55.125 INFO:tasks.cephadm:setting config [mds] mds_bal_split_size = 100 2024-09-21T02:51:55.126 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_bal_split_size 100 2024-09-21T02:51:55.137 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:55.155 INFO:tasks.cephadm:setting config [mds] mds_debug_frag = True 2024-09-21T02:51:55.155 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_frag True 2024-09-21T02:51:55.199 INFO:tasks.cephadm:setting config [mds] mds_debug_scatterstat = True 2024-09-21T02:51:55.200 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_debug_scatterstat True 2024-09-21T02:51:55.249 INFO:tasks.cephadm:setting config [mds] mds_op_complaint_time = 180 2024-09-21T02:51:55.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_op_complaint_time 180 2024-09-21T02:51:55.314 INFO:tasks.cephadm:setting config [mds] mds_verify_scatter = True 2024-09-21T02:51:55.315 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds mds_verify_scatter True 2024-09-21T02:51:55.360 INFO:tasks.cephadm:setting config [mds] osd_op_complaint_time = 180 2024-09-21T02:51:55.360 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds osd_op_complaint_time 180 2024-09-21T02:51:55.417 INFO:tasks.cephadm:setting config [mds] rados_mon_op_timeout = 900 2024-09-21T02:51:55.418 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds rados_mon_op_timeout 900 2024-09-21T02:51:55.468 INFO:tasks.cephadm:setting config [mds] rados_osd_op_timeout = 900 2024-09-21T02:51:55.468 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mds rados_osd_op_timeout 900 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:51:55.353+0000 7f4f91d38640 -1 log_channel(cephadm) log [ERR] : [21/Sep/2024:02:51:55] ENGINE Error in HTTPServer.serve 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: Traceback (most recent call last): 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._connections.run(self.expiration_interval) 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:51:55.494 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._run(expiration_interval) 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: s = self.context.wrap_socket( 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: return self.sslsocket_class._create( 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self.do_handshake() 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: self._sslobj.do_handshake() 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:51:55.495 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:51:55.503 INFO:tasks.cephadm:setting config [mgr] client_mount_timeout = 30 2024-09-21T02:51:55.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr client_mount_timeout 30 2024-09-21T02:51:55.548 INFO:tasks.cephadm:setting config [mgr] debug_cephsqlite = 20 2024-09-21T02:51:55.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr debug_cephsqlite 20 2024-09-21T02:51:55.602 INFO:tasks.cephadm:setting config [mgr] debug_client = 20 2024-09-21T02:51:55.603 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr debug_client 20 2024-09-21T02:51:55.647 INFO:tasks.cephadm:setting config [mgr] debug_mgr = 20 2024-09-21T02:51:55.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr debug_mgr 20 2024-09-21T02:51:55.700 INFO:tasks.cephadm:setting config [mgr] debug_ms = 1 2024-09-21T02:51:55.700 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr debug_ms 1 2024-09-21T02:51:55.756 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/allow_m_granularity = True 2024-09-21T02:51:55.756 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2024-09-21T02:51:55.812 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/dump_on_update = True 2024-09-21T02:51:55.813 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/dump_on_update True 2024-09-21T02:51:55.862 INFO:tasks.cephadm:setting config [mgr] mgr/snap_schedule/log_level = debug 2024-09-21T02:51:55.862 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr mgr/snap_schedule/log_level debug 2024-09-21T02:51:55.911 INFO:tasks.cephadm:setting config [mgr] mon_warn_on_pool_no_app = False 2024-09-21T02:51:55.911 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mgr mon_warn_on_pool_no_app False 2024-09-21T02:51:55.968 INFO:tasks.cephadm:setting config [mon] mon_op_complaint_time = 120 2024-09-21T02:51:55.969 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set mon mon_op_complaint_time 120 2024-09-21T02:51:56.018 INFO:tasks.cephadm:setting config [osd] osd_op_complaint_time = 180 2024-09-21T02:51:56.019 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config set osd osd_op_complaint_time 180 2024-09-21T02:51:56.069 DEBUG:tasks.cephadm:set 30 configs 2024-09-21T02:51:56.070 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'client_mount_timeout', '600'], check_status=True, hostname='smithi016') 2024-09-21T02:51:56.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: Saving service mgr spec with placement smithi016=x;smithi018=y;smithi123=z;count:3 2024-09-21T02:51:56.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.164 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.165 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:56.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: Saving service mgr spec with placement smithi016=x;smithi018=y;smithi123=z;count:3 2024-09-21T02:51:56.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:56.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: Saving service mgr spec with placement smithi016=x;smithi018=y;smithi123=z;count:3 2024-09-21T02:51:56.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.223 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:56.223 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi018", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: [21/Sep/2024:02:51:55] ENGINE Bus STARTING 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: [21/Sep/2024:02:51:55] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: [21/Sep/2024:02:51:55] ENGINE Error in HTTPServer.serve 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout: Traceback (most recent call last): 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:51:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._run(expiration_interval) 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: s = self.context.wrap_socket( 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: return self.sslsocket_class._create( 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: self.do_handshake() 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: self._sslobj.do_handshake() 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:51:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: [21/Sep/2024:02:51:55] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: [21/Sep/2024:02:51:55] ENGINE Bus STARTED 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: mgrmap e17: x(active, since 2s) 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2531122525' entity='client.admin' 2024-09-21T02:51:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:56 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:51:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: [21/Sep/2024:02:51:55] ENGINE Bus STARTING 2024-09-21T02:51:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: [21/Sep/2024:02:51:55] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T02:51:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: [21/Sep/2024:02:51:55] ENGINE Error in HTTPServer.serve 2024-09-21T02:51:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout: Traceback (most recent call last): 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: self._run(expiration_interval) 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: s = self.context.wrap_socket( 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: return self.sslsocket_class._create( 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:51:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout: self.do_handshake() 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout: self._sslobj.do_handshake() 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: [21/Sep/2024:02:51:55] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: [21/Sep/2024:02:51:55] ENGINE Bus STARTED 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: mgrmap e17: x(active, since 2s) 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2531122525' entity='client.admin' 2024-09-21T02:51:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:56 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: [21/Sep/2024:02:51:55] ENGINE Bus STARTING 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: [21/Sep/2024:02:51:55] ENGINE Serving on https://172.21.15.16:7150 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: [21/Sep/2024:02:51:55] ENGINE Error in HTTPServer.serve 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: Traceback (most recent call last): 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/server.py", line 1823, in serve 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._connections.run(self.expiration_interval) 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 203, in run 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._run(expiration_interval) 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 246, in _run 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: new_conn = self._from_server_socket(self.server.socket) 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/connections.py", line 300, in _from_server_socket 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: s, ssl_env = self.server.ssl_adapter.wrap(s) 2024-09-21T02:51:57.221 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib/python3.9/site-packages/cheroot/ssl/builtin.py", line 277, in wrap 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: s = self.context.wrap_socket( 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 501, in wrap_socket 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: return self.sslsocket_class._create( 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1074, in _create 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: self.do_handshake() 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: File "/lib64/python3.9/ssl.py", line 1343, in do_handshake 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: self._sslobj.do_handshake() 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout: ssl.SSLZeroReturnError: TLS/SSL connection has been closed (EOF) (_ssl.c:1133) 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: [21/Sep/2024:02:51:55] ENGINE Serving on http://172.21.15.16:8765 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: [21/Sep/2024:02:51:55] ENGINE Bus STARTED 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi123", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:57.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: mgrmap e17: x(active, since 2s) 2024-09-21T02:51:57.223 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2531122525' entity='client.admin' 2024-09-21T02:51:57.223 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:56 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:51:57.405 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:51:57.979 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-21T02:51:57.988 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.123 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.281 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.282 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.284 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.288 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.294 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.296 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.299 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:51:58.658 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.736 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:51:58.737 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd/host:smithi016", "name": "osd_memory_target"} : dispatch 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi016:/etc/ceph/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi018:/etc/ceph/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi123:/etc/ceph/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.conf 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi016:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi018:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: Updating smithi123:/etc/ceph/ceph.client.admin.keyring 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:51:58.934 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f448803fb70 msgr2=0x7f4488042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f448803fb70 0x7f4488042080 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f4498005d00 tx=0x7f449800b410 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c109e10 msgr2=0x7f449c19fab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c109e10 0x7f449c19fab0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4494005cc0 tx=0x7f4494003c20 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 shutdown_connections 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f448803fb70 0x7f4488042080 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f449c109e10 0x7f449c19fab0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f449c1094c0 0x7f449c19a810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 --2- 172.21.15.16:0/1755924244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f449c103b80 0x7f449c19a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 >> 172.21.15.16:0/1755924244 conn(0x7f449c0ff610 msgr2=0x7f449c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:51:59.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 shutdown_connections 2024-09-21T02:51:59.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:51:59.196+0000 7f44a3361640 1 -- 172.21.15.16:0/1755924244 wait complete. 2024-09-21T02:51:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:59 smithi016 ceph-mon[31253]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:59 smithi016 ceph-mon[31253]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:59 smithi016 ceph-mon[31253]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:59 smithi016 ceph-mon[31253]: Deploying daemon mgr.z on smithi123 2024-09-21T02:51:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:51:59 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1755924244' entity='client.admin' 2024-09-21T02:51:59.810 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:59 smithi123 ceph-mon[34645]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.811 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:59 smithi123 ceph-mon[34645]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.811 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:59 smithi123 ceph-mon[34645]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.811 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:59 smithi123 ceph-mon[34645]: Deploying daemon mgr.z on smithi123 2024-09-21T02:51:59.811 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:51:59 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1755924244' entity='client.admin' 2024-09-21T02:51:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:59 smithi018 ceph-mon[34587]: Updating smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:59 smithi018 ceph-mon[34587]: Updating smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:59 smithi018 ceph-mon[34587]: Updating smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/config/ceph.client.admin.keyring 2024-09-21T02:51:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:59 smithi018 ceph-mon[34587]: Deploying daemon mgr.z on smithi123 2024-09-21T02:51:59.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:51:59 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1755924244' entity='client.admin' 2024-09-21T02:52:00.470 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 21 02:52:00 smithi123 systemd[1]: Started Ceph mgr.z for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:52:00.536 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.032+0000 7fc2b4585640 1 -- 172.21.15.16:0/2332752933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac103b60 msgr2=0x7fc2ac103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.032+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2332752933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac103b60 0x7fc2ac103f70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8009900 tx=0x7fc2a802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 -- 172.21.15.16:0/2332752933 shutdown_connections 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2332752933 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac109df0 0x7fc2ac110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2332752933 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2332752933 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac103b60 0x7fc2ac103f70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 -- 172.21.15.16:0/2332752933 >> 172.21.15.16:0/2332752933 conn(0x7fc2ac07b850 msgr2=0x7fc2ac07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 -- 172.21.15.16:0/2332752933 shutdown_connections 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 -- 172.21.15.16:0/2332752933 wait complete. 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 Processor -- start 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.033+0000 7fc2b4585640 1 -- start start 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac103b60 0x7fc2ac075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac113870 con 0x7fc2ac109df0 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1136f0 con 0x7fc2ac103b60 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b4585640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc2ac1139f0 con 0x7fc2ac1094a0 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34170/0 (socket says 172.21.15.16:34170) 2024-09-21T02:52:01.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 -- 172.21.15.16:0/2736018285 learned_addr learned my addr 172.21.15.16:0/2736018285 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b1b20640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2321640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac103b60 0x7fc2ac075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 -- 172.21.15.16:0/2736018285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 msgr2=0x7fc2ac075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 -- 172.21.15.16:0/2736018285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac103b60 msgr2=0x7fc2ac075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac103b60 0x7fc2ac075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 -- 172.21.15.16:0/2736018285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2ac1a8660 con 0x7fc2ac109df0 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b1b20640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.034+0000 7fc2b2b22640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400eb10 tx=0x7fc2a400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.035+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4004b20 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.035+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fc2a4004cc0 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.035+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2ac1a8950 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.035+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2a4010690 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.035+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2ac1a8e90 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.036+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc2ac103f70 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.036+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fc2a4010830 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.036+0000 7fc2a37fe640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 0x7fc298042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.036+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fc2a4054c30 con 0x7fc2ac109df0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.036+0000 7fc2b2321640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 0x7fc298042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.037+0000 7fc2b2321640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 0x7fc298042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8000c00 tx=0x7fc2a8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.041+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc2a40059f0 con 0x7fc2ac109df0 2024-09-21T02:52:01.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.230+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) -- 0x7fc2ac10bb70 con 0x7fc2ac109df0 2024-09-21T02:52:01.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.234+0000 7fc2a37fe640 1 -- 172.21.15.16:0/2736018285 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v13)=0 v13) ==== 108+0+0 (secure 0 0 0) 0x7fc2a4019e20 con 0x7fc2ac109df0 2024-09-21T02:52:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 msgr2=0x7fc298042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 0x7fc298042080 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc2a8000c00 tx=0x7fc2a8031400 comp rx=0 tx=0).stop 2024-09-21T02:52:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 msgr2=0x7fc2ac076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fc2a400eb10 tx=0x7fc2a400c2d0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 shutdown_connections 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc29803fb70 0x7fc298042080 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc2ac109df0 0x7fc2ac076190 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc2ac1094a0 0x7fc2ac075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 --2- 172.21.15.16:0/2736018285 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc2ac103b60 0x7fc2ac075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 >> 172.21.15.16:0/2736018285 conn(0x7fc2ac07b850 msgr2=0x7fc2ac107700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 shutdown_connections 2024-09-21T02:52:01.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.237+0000 7fc2b4585640 1 -- 172.21.15.16:0/2736018285 wait complete. 2024-09-21T02:52:01.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 -- 172.21.15.16:0/2183516408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0108d00 msgr2=0x7fd4f010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 --2- 172.21.15.16:0/2183516408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0108d00 0x7fd4f010f530 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fd4e8009ca0 tx=0x7fd4e802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 -- 172.21.15.16:0/2183516408 shutdown_connections 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 --2- 172.21.15.16:0/2183516408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0108d00 0x7fd4f010f530 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 --2- 172.21.15.16:0/2183516408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 0x7fd4f01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 --2- 172.21.15.16:0/2183516408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0102a70 0x7fd4f0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 -- 172.21.15.16:0/2183516408 >> 172.21.15.16:0/2183516408 conn(0x7fd4f0079730 msgr2=0x7fd4f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 -- 172.21.15.16:0/2183516408 shutdown_connections 2024-09-21T02:52:01.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.353+0000 7fd4f4933640 1 -- 172.21.15.16:0/2183516408 wait complete. 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 Processor -- start 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 -- start start 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 0x7fd4f019b690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0108d00 0x7fd4f019bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f01126e0 con 0x7fd4f0102a70 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0112560 con 0x7fd4f0108d00 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4f4933640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd4f0112860 con 0x7fd4f01083b0 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4ee59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34190/0 (socket says 172.21.15.16:34190) 2024-09-21T02:52:01.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4ee59c640 1 -- 172.21.15.16:0/969266094 learned_addr learned my addr 172.21.15.16:0/969266094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4edd9b640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 0x7fd4f019b690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.354+0000 7fd4eed9d640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0108d00 0x7fd4f019bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 -- 172.21.15.16:0/969266094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 msgr2=0x7fd4f019b690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 0x7fd4f019b690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 -- 172.21.15.16:0/969266094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0108d00 msgr2=0x7fd4f019bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0108d00 0x7fd4f019bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 -- 172.21.15.16:0/969266094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd4f01a63e0 con 0x7fd4f0102a70 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4ee59c640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e9f0 tx=0x7fd4e000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e000cd80 con 0x7fd4f0102a70 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7fd4e00108a0 con 0x7fd4f0102a70 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd4f01a66d0 con 0x7fd4f0102a70 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd4e0018760 con 0x7fd4f0102a70 2024-09-21T02:52:01.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.355+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd4f01a6c60 con 0x7fd4f0102a70 2024-09-21T02:52:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.356+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd4f0103cf0 con 0x7fd4f0102a70 2024-09-21T02:52:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.356+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fd4e0018900 con 0x7fd4f0102a70 2024-09-21T02:52:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.356+0000 7fd4df7fe640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 0x7fd4d80420b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.356+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fd4e0054a00 con 0x7fd4f0102a70 2024-09-21T02:52:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.357+0000 7fd4edd9b640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 0x7fd4d80420b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.357+0000 7fd4edd9b640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 0x7fd4d80420b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0079b80 tx=0x7fd4e400b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.361+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd4e00071c0 con 0x7fd4f0102a70 2024-09-21T02:52:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: Deploying daemon mgr.y on smithi018 2024-09-21T02:52:01.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:01 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2736018285' entity='client.admin' 2024-09-21T02:52:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: Deploying daemon mgr.y on smithi018 2024-09-21T02:52:01.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:01 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2736018285' entity='client.admin' 2024-09-21T02:52:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 -- 172.21.15.16:0/1750439451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 msgr2=0x7f199c10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/1750439451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c10d850 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f1990009900 tx=0x7f199002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 -- 172.21.15.16:0/1750439451 shutdown_connections 2024-09-21T02:52:01.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/1750439451 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c10d850 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/1750439451 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 0x7f199c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/1750439451 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 -- 172.21.15.16:0/1750439451 >> 172.21.15.16:0/1750439451 conn(0x7f199c079730 msgr2=0x7f199c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 -- 172.21.15.16:0/1750439451 shutdown_connections 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.539+0000 7f19a1e1a640 1 -- 172.21.15.16:0/1750439451 wait complete. 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 Processor -- start 2024-09-21T02:52:01.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 -- start start 2024-09-21T02:52:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c19a350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c19a8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 0x7f199c19fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.540+0000 7f19a1e1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f199c113880 con 0x7f199c105300 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f19a1e1a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f199c113700 con 0x7f199c1049b0 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f19a1e1a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f199c113a00 con 0x7f199c10de10 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f199affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c19a8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f199bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 0x7f199c19fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f199affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c19a8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34198/0 (socket says 172.21.15.16:34198) 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f199affd640 1 -- 172.21.15.16:0/3816674517 learned_addr learned my addr 172.21.15.16:0/3816674517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:01.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.541+0000 7f199b7fe640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c19a350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 -- 172.21.15.16:0/3816674517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 msgr2=0x7f199c19fb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 0x7f199c19fb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 -- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 msgr2=0x7f199c19a8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c19a8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 -- 172.21.15.16:0/3816674517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f199c1a02c0 con 0x7f199c1049b0 2024-09-21T02:52:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f199b7fe640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c19a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9c0 tx=0x7f198c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c00ccc0 con 0x7f199c1049b0 2024-09-21T02:52:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(24 keys) ==== 1039+0+0 (secure 0 0 0) 0x7f198c004490 con 0x7f199c1049b0 2024-09-21T02:52:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f199c1a05b0 con 0x7f199c1049b0 2024-09-21T02:52:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f198c003d60 con 0x7f199c1049b0 2024-09-21T02:52:01.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.542+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f199c1a6d10 con 0x7f199c1049b0 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.544+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f198c018430 con 0x7f199c1049b0 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.544+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f199c104dc0 con 0x7f199c1049b0 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.544+0000 7f1998ff9640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 0x7f1984042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.544+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f198c055700 con 0x7f199c1049b0 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.544+0000 7f199affd640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 0x7f1984042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.545+0000 7f199affd640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 0x7f1984042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1990006010 tx=0x7f1990005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.549+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f198c021430 con 0x7f199c1049b0 2024-09-21T02:52:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.558+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_mgr}] v 0) -- 0x7fd4f010aaf0 con 0x7fd4f0102a70 2024-09-21T02:52:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.558+0000 7fd4df7fe640 1 -- 172.21.15.16:0/969266094 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mgr}]=0 v13)=0 v13) ==== 108+0+0 (secure 0 0 0) 0x7fd4e001fe00 con 0x7fd4f0102a70 2024-09-21T02:52:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 msgr2=0x7fd4d80420b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 0x7fd4d80420b0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd4f0079b80 tx=0x7fd4e400b410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 msgr2=0x7fd4f019b150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fd4e000e9f0 tx=0x7fd4e000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 shutdown_connections 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd4d803fba0 0x7fd4d80420b0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd4f0108d00 0x7fd4f019bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4f01083b0 0x7fd4f019b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 --2- 172.21.15.16:0/969266094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd4f0102a70 0x7fd4f019b150 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 >> 172.21.15.16:0/969266094 conn(0x7fd4f0079730 msgr2=0x7fd4f0104520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 shutdown_connections 2024-09-21T02:52:01.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.561+0000 7fd4f4933640 1 -- 172.21.15.16:0/969266094 wait complete. 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: Deploying daemon mgr.y on smithi018 2024-09-21T02:52:01.642 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:01 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2736018285' entity='client.admin' 2024-09-21T02:52:01.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.741+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7f199c108f10 con 0x7f199c1049b0 2024-09-21T02:52:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.746+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 7 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f198c0035f0 con 0x7f199c1049b0 2024-09-21T02:52:01.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.747+0000 7f1998ff9640 1 -- 172.21.15.16:0/3816674517 <== mon.2 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v14)=0 v14) ==== 123+0+0 (secure 0 0 0) 0x7f198c0217b0 con 0x7f199c1049b0 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.748+0000 7fee53c5c640 1 -- 172.21.15.16:0/2692417073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c103b90 msgr2=0x7fee4c103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.748+0000 7fee53c5c640 1 --2- 172.21.15.16:0/2692417073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c103b90 0x7fee4c103fa0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fee48009900 tx=0x7fee4802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 -- 172.21.15.16:0/2692417073 shutdown_connections 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 --2- 172.21.15.16:0/2692417073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c109e20 0x7fee4c110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 --2- 172.21.15.16:0/2692417073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c1094d0 0x7fee4c1098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 --2- 172.21.15.16:0/2692417073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c103b90 0x7fee4c103fa0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 -- 172.21.15.16:0/2692417073 >> 172.21.15.16:0/2692417073 conn(0x7fee4c0ff600 msgr2=0x7fee4c101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 -- 172.21.15.16:0/2692417073 shutdown_connections 2024-09-21T02:52:01.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7fee53c5c640 1 -- 172.21.15.16:0/2692417073 wait complete. 2024-09-21T02:52:01.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 msgr2=0x7f1984042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 0x7f1984042080 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f1990006010 tx=0x7f1990005a20 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 msgr2=0x7f199c19a350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c19a350 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f198c00e9c0 tx=0x7f198c00ee80 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 shutdown_connections 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198403fb70 0x7f1984042080 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f199c105300 0x7f199c19a8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f199c10de10 0x7f199c19fb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 --2- 172.21.15.16:0/3816674517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f199c1049b0 0x7f199c19a350 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.749+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 >> 172.21.15.16:0/3816674517 conn(0x7f199c079730 msgr2=0x7f199c0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 shutdown_connections 2024-09-21T02:52:01.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7f19a1e1a640 1 -- 172.21.15.16:0/3816674517 wait complete. 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 Processor -- start 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 -- start start 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b90 0x7fee4c077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 0x7fee4c075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113a10 con 0x7fee4c1094d0 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113890 con 0x7fee4c109e20 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee53c5c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fee4c113b90 con 0x7fee4c103b90 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee511f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee511f7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34224/0 (socket says 172.21.15.16:34224) 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee511f7640 1 -- 172.21.15.16:0/3751579470 learned_addr learned my addr 172.21.15.16:0/3751579470 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:01.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee519f8640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b90 0x7fee4c077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.750+0000 7fee521f9640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 0x7fee4c075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 -- 172.21.15.16:0/3751579470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b90 msgr2=0x7fee4c077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b90 0x7fee4c077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 -- 172.21.15.16:0/3751579470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 msgr2=0x7fee4c075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 0x7fee4c075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 -- 172.21.15.16:0/3751579470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee4c076470 con 0x7fee4c1094d0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee521f9640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 0x7fee4c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee511f7640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d950 tx=0x7fee3c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c017070 con 0x7fee4c1094d0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee4c1acd80 con 0x7fee4c1094d0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee4c1ad240 con 0x7fee4c1094d0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fee3c009c30 con 0x7fee4c1094d0 2024-09-21T02:52:01.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.751+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee3c0047a0 con 0x7fee4c1094d0 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.752+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fee3c004940 con 0x7fee4c1094d0 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.753+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fee300050b0 con 0x7fee4c1094d0 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.753+0000 7fee42ffd640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 0x7fee38042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.753+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fee3c055100 con 0x7fee4c1094d0 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.753+0000 7fee519f8640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 0x7fee38042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:01.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.753+0000 7fee519f8640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 0x7fee38042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee48006010 tx=0x7fee48000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:01.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.758+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee3c020020 con 0x7fee4c1094d0 2024-09-21T02:52:01.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.949+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_osd_op_timeout}] v 0) -- 0x7fee300053a0 con 0x7fee4c1094d0 2024-09-21T02:52:01.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.953+0000 7fee42ffd640 1 -- 172.21.15.16:0/3751579470 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_osd_op_timeout}]=0 v15)=0 v15) ==== 120+0+0 (secure 0 0 0) 0x7fee3c009de0 con 0x7fee4c1094d0 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 msgr2=0x7fee38042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 0x7fee38042080 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7fee48006010 tx=0x7fee48000bc0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 msgr2=0x7fee4c075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00d950 tx=0x7fee3c00de10 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 shutdown_connections 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee3803fb70 0x7fee38042080 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fee4c1094d0 0x7fee4c075710 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fee4c109e20 0x7fee4c075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 --2- 172.21.15.16:0/3751579470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fee4c103b90 0x7fee4c077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 >> 172.21.15.16:0/3751579470 conn(0x7fee4c0ff600 msgr2=0x7fee4c101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 shutdown_connections 2024-09-21T02:52:01.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:01.955+0000 7fee53c5c640 1 -- 172.21.15.16:0/3751579470 wait complete. 2024-09-21T02:52:02.321 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 systemd[1]: Starting Ceph mgr.y for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 podman[36091]: 2024-09-21 02:52:02.320066356 +0000 UTC m=+0.094640802 container create fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 podman[36091]: 2024-09-21 02:52:02.241124396 +0000 UTC m=+0.015698849 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 podman[36091]: 2024-09-21 02:52:02.514082345 +0000 UTC m=+0.288656786 container init fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, ceph=True) 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 podman[36091]: 2024-09-21 02:52:02.516988522 +0000 UTC m=+0.291562970 container start fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True) 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mgr[36110]: -- 172.21.15.18:0/988017949 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x561ecb39f0e0 con 0x561ed0142000 2024-09-21T02:52:02.615 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 bash[36091]: fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b 2024-09-21T02:52:02.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:52:02.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3751579470' entity='client.admin' 2024-09-21T02:52:02.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 02:52:02 smithi018 systemd[1]: Started Ceph mgr.y for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3751579470' entity='client.admin' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:52:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3751579470' entity='client.admin' 2024-09-21T02:52:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1159521019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 msgr2=0x7fa3f010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1159521019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f010f520 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8009ca0 tx=0x7fa3e802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1159521019 shutdown_connections 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1159521019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f010f520 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1159521019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1159521019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1159521019 >> 172.21.15.16:0/1159521019 conn(0x7fa3f0079730 msgr2=0x7fa3f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1159521019 shutdown_connections 2024-09-21T02:52:03.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.793+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1159521019 wait complete. 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 Processor -- start 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 -- start start 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f0199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:03.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f0112840 con 0x7fa3f0108cf0 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01126c0 con 0x7fa3f01083a0 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f81ea640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3f01129c0 con 0x7fa3f0102a60 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f5f86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f5f86640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45698/0 (socket says 172.21.15.16:45698) 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f6787640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f0199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f5f86640 1 -- 172.21.15.16:0/1379108855 learned_addr learned my addr 172.21.15.16:0/1379108855 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:03.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.794+0000 7fa3f5785640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 -- 172.21.15.16:0/1379108855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 msgr2=0x7fa3f0198fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 -- 172.21.15.16:0/1379108855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 msgr2=0x7fa3f01994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 -- 172.21.15.16:0/1379108855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3f01a8650 con 0x7fa3f0108cf0 2024-09-21T02:52:03.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f5f86640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f5785640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f6787640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f0199a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005e30 tx=0x7fa3e8009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8003c30 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa3e8003dd0 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3f01a88e0 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.795+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3e8030a90 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.796+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3f01a8e70 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.797+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7fa3e8030c30 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.797+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3f0102e70 con 0x7fa3f0108cf0 2024-09-21T02:52:03.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.797+0000 7fa3e6ffd640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 0x7fa3dc042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.797+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fa3e8075730 con 0x7fa3f0108cf0 2024-09-21T02:52:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.797+0000 7fa3f5f86640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 0x7fa3dc042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:03.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.798+0000 7fa3f5f86640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 0x7fa3dc042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0107ae0 tx=0x7fa3ec005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:03.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.802+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa3e80412e0 con 0x7fa3f0108cf0 2024-09-21T02:52:03.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:03.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:03.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:03.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:03.999+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_bits}] v 0) -- 0x7fa3f010aae0 con 0x7fa3f0108cf0 2024-09-21T02:52:04.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.002+0000 7fa3e6ffd640 1 -- 172.21.15.16:0/1379108855 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_bits}]=0 v16)=0 v16) ==== 116+0+0 (secure 0 0 0) 0x7fa3e803ad20 con 0x7fa3f0108cf0 2024-09-21T02:52:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 msgr2=0x7fa3dc042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 0x7fa3dc042080 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fa3f0107ae0 tx=0x7fa3ec005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 msgr2=0x7fa3f0199a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f0199a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa3e8005e30 tx=0x7fa3e8009800 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 shutdown_connections 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3dc03fb70 0x7fa3dc042080 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3f0108cf0 0x7fa3f0199a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa3f01083a0 0x7fa3f01994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 --2- 172.21.15.16:0/1379108855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3f0102a60 0x7fa3f0198fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 >> 172.21.15.16:0/1379108855 conn(0x7fa3f0079730 msgr2=0x7fa3f0104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 shutdown_connections 2024-09-21T02:52:04.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.005+0000 7fa3f81ea640 1 -- 172.21.15.16:0/1379108855 wait complete. 2024-09-21T02:52:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 -- 172.21.15.16:0/2166567795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c1031d0 msgr2=0x7f191c1035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 --2- 172.21.15.16:0/2166567795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c1031d0 0x7f191c1035e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f1910009900 tx=0x7f191002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 -- 172.21.15.16:0/2166567795 shutdown_connections 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 --2- 172.21.15.16:0/2166567795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c108f60 0x7f191c10f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 --2- 172.21.15.16:0/2166567795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108610 0x7f191c108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 --2- 172.21.15.16:0/2166567795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c1031d0 0x7f191c1035e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 -- 172.21.15.16:0/2166567795 >> 172.21.15.16:0/2166567795 conn(0x7f191c079e40 msgr2=0x7f191c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 -- 172.21.15.16:0/2166567795 shutdown_connections 2024-09-21T02:52:04.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.572+0000 7f1922f39640 1 -- 172.21.15.16:0/2166567795 wait complete. 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 Processor -- start 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 -- start start 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c1031d0 0x7f191c1128e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108f60 0x7f191c113360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f191c1021c0 con 0x7f191c108610 2024-09-21T02:52:04.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f191c102040 con 0x7f191c1031d0 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1922f39640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f191c102340 con 0x7f191c108f60 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f191bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f1920cd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c1031d0 0x7f191c1128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f191bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33016/0 (socket says 172.21.15.16:33016) 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f191bfff640 1 -- 172.21.15.16:0/750725178 learned_addr learned my addr 172.21.15.16:0/750725178 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.573+0000 7f19214d6640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108f60 0x7f191c113360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 -- 172.21.15.16:0/750725178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108f60 msgr2=0x7f191c113360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108f60 0x7f191c113360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 -- 172.21.15.16:0/750725178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c1031d0 msgr2=0x7f191c1128e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c1031d0 0x7f191c1128e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 -- 172.21.15.16:0/750725178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f191c1a88c0 con 0x7f191c108610 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f191bfff640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9c0 tx=0x7f190800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:04.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f190800ccc0 con 0x7f191c108610 2024-09-21T02:52:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f19080054b0 con 0x7f191c108610 2024-09-21T02:52:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f191c1a8bb0 con 0x7f191c108610 2024-09-21T02:52:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1908010430 con 0x7f191c108610 2024-09-21T02:52:04.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.574+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f191c1a90f0 con 0x7f191c108610 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.575+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f191c1035e0 con 0x7f191c108610 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.575+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f19080105d0 con 0x7f191c108610 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.576+0000 7f1919ffb640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 0x7f190c042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.576+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1908054920 con 0x7f191c108610 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.576+0000 7f1920cd5640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 0x7f190c042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.576+0000 7f1920cd5640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 0x7f190c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1910006010 tx=0x7f1910000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:04.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.581+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1908016330 con 0x7f191c108610 2024-09-21T02:52:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.762+0000 7feba6f79640 1 -- 172.21.15.16:0/2009757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 msgr2=0x7feba010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.762+0000 7feba6f79640 1 --2- 172.21.15.16:0/2009757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba010f520 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7feb98009ca0 tx=0x7feb9802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 -- 172.21.15.16:0/2009757071 shutdown_connections 2024-09-21T02:52:04.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 --2- 172.21.15.16:0/2009757071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba010f520 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 --2- 172.21.15.16:0/2009757071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba01083a0 0x7feba01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 --2- 172.21.15.16:0/2009757071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba0102a60 0x7feba0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 -- 172.21.15.16:0/2009757071 >> 172.21.15.16:0/2009757071 conn(0x7feba0079730 msgr2=0x7feba0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 -- 172.21.15.16:0/2009757071 shutdown_connections 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 -- 172.21.15.16:0/2009757071 wait complete. 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 Processor -- start 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.763+0000 7feba6f79640 1 -- start start 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba01a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feba0112840 con 0x7feba0108cf0 2024-09-21T02:52:04.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feba01126c0 con 0x7feba0102a60 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba6f79640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feba01129c0 con 0x7feba01083a0 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba4d15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba01a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba4d15640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:50374/0 (socket says 172.21.15.16:50374) 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba4d15640 1 -- 172.21.15.16:0/1407655403 learned_addr learned my addr 172.21.15.16:0/1407655403 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feb9ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49170/0 (socket says 172.21.15.16:49170) 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 -- 172.21.15.16:0/1407655403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 msgr2=0x7feba019caf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 -- 172.21.15.16:0/1407655403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 msgr2=0x7feba019c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 -- 172.21.15.16:0/1407655403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feba01a24c0 con 0x7feba0108cf0 2024-09-21T02:52:04.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feb9ffff640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba4d15640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.764+0000 7feba5516640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba01a1d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb98005e30 tx=0x7feb98009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.765+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98003c30 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.765+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feba01a2750 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.765+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7feb98003dd0 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.765+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb98030a90 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.765+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feba01a8f10 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.766+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feba0102e70 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.766+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7feb98030c50 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.766+0000 7feb9dffb640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 0x7feb90042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.766+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7feb98075460 con 0x7feba0108cf0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.767+0000 7feba4d15640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 0x7feb90042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:04.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.767+0000 7feba4d15640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 0x7feb90042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb94011550 tx=0x7feb94005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:04.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.771+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feb98039e70 con 0x7feba0108cf0 2024-09-21T02:52:04.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.773+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_scatterstat}] v 0) -- 0x7f191c06b960 con 0x7f191c108610 2024-09-21T02:52:04.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.776+0000 7f1919ffb640 1 -- 172.21.15.16:0/750725178 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_scatterstat}]=0 v17)=0 v17) ==== 122+0+0 (secure 0 0 0) 0x7f19080046c0 con 0x7f191c108610 2024-09-21T02:52:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 msgr2=0x7f190c042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 0x7f190c042080 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f1910006010 tx=0x7f1910000bc0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 msgr2=0x7f191c112e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f190800e9c0 tx=0x7f190800ee80 comp rx=0 tx=0).stop 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 shutdown_connections 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f190c03fb70 0x7f190c042080 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f191c108610 0x7f191c112e20 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f191c108f60 0x7f191c113360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 --2- 172.21.15.16:0/750725178 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f191c1031d0 0x7f191c1128e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 >> 172.21.15.16:0/750725178 conn(0x7f191c079e40 msgr2=0x7f191c100d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 shutdown_connections 2024-09-21T02:52:04.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.779+0000 7f1922f39640 1 -- 172.21.15.16:0/750725178 wait complete. 2024-09-21T02:52:04.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.965+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_warn_on_pool_no_app}] v 0) -- 0x7feba01a90e0 con 0x7feba0108cf0 2024-09-21T02:52:04.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1379108855' entity='client.admin' 2024-09-21T02:52:04.968 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/750725178' entity='client.admin' 2024-09-21T02:52:04.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.969+0000 7feb9dffb640 1 -- 172.21.15.16:0/1407655403 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_warn_on_pool_no_app}]=0 v18)=0 v18) ==== 125+0+0 (secure 0 0 0) 0x7feb98037e00 con 0x7feba0108cf0 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 msgr2=0x7feb90042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 0x7feb90042080 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7feb94011550 tx=0x7feb94005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 msgr2=0x7feba01a1d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba01a1d90 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7feb98005e30 tx=0x7feb98009800 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 shutdown_connections 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feb9003fb70 0x7feb90042080 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feba0108cf0 0x7feba01a1d90 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feba01083a0 0x7feba019caf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 --2- 172.21.15.16:0/1407655403 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feba0102a60 0x7feba019c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 >> 172.21.15.16:0/1407655403 conn(0x7feba0079730 msgr2=0x7feba006afa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 shutdown_connections 2024-09-21T02:52:04.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.972+0000 7feba6f79640 1 -- 172.21.15.16:0/1407655403 wait complete. 2024-09-21T02:52:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 -- 172.21.15.16:0/2103273761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 msgr2=0x7f3ec81087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 --2- 172.21.15.16:0/2103273761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec81087b0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc009900 tx=0x7f3ebc02a410 comp rx=0 tx=0).stop 2024-09-21T02:52:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 -- 172.21.15.16:0/2103273761 shutdown_connections 2024-09-21T02:52:05.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 --2- 172.21.15.16:0/2103273761 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec81087b0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 --2- 172.21.15.16:0/2103273761 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 0x7f3ec810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 --2- 172.21.15.16:0/2103273761 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 -- 172.21.15.16:0/2103273761 >> 172.21.15.16:0/2103273761 conn(0x7f3ec8079730 msgr2=0x7f3ec8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 -- 172.21.15.16:0/2103273761 shutdown_connections 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.096+0000 7f3ece270640 1 -- 172.21.15.16:0/2103273761 wait complete. 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 Processor -- start 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 -- start start 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec819c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 0x7f3ec81a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8113960 con 0x7f3ec81083a0 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec81137e0 con 0x7f3ec8108cf0 2024-09-21T02:52:05.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.097+0000 7f3ece270640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3ec8113ae0 con 0x7f3ec8102a60 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33070/0 (socket says 172.21.15.16:33070) 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec77fe640 1 -- 172.21.15.16:0/1367315360 learned_addr learned my addr 172.21.15.16:0/1367315360 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec819c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ecc80d640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 0x7f3ec81a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 -- 172.21.15.16:0/1367315360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 msgr2=0x7f3ec81a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 0x7f3ec81a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 -- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 msgr2=0x7f3ec819cc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 -- 172.21.15.16:0/1367315360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ec81a25e0 con 0x7f3ec8102a60 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec77fe640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec7fff640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec819c670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9f0 tx=0x7f3eb800eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:05.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800ccc0 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f3eb8005020 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ec81a28d0 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3eb800ce60 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.098+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ec81a9030 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.099+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f3eb8004a60 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.100+0000 7f3ec57fa640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 0x7f3eb0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.100+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f3eb80544f0 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.100+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb40050b0 con 0x7f3ec8102a60 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.100+0000 7f3ec77fe640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 0x7f3eb0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.100+0000 7f3ec77fe640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 0x7f3eb0042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0096c0 tx=0x7f3ebc0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:05.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.105+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3eb8014230 con 0x7f3ec8102a60 2024-09-21T02:52:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1379108855' entity='client.admin' 2024-09-21T02:52:05.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/750725178' entity='client.admin' 2024-09-21T02:52:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: Reconfiguring mgr.x (unknown last config time)... 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]} : dispatch 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr services"} : dispatch 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: Reconfiguring daemon mgr.x on smithi016 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1379108855' entity='client.admin' 2024-09-21T02:52:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/750725178' entity='client.admin' 2024-09-21T02:52:05.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.298+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mds_bal_merge_size}] v 0) -- 0x7f3eb40053a0 con 0x7f3ec8102a60 2024-09-21T02:52:05.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.305+0000 7f3ec57fa640 1 -- 172.21.15.16:0/1367315360 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_merge_size}]=0 v20)=0 v20) ==== 116+0+0 (secure 0 0 0) 0x7f3eb800fdd0 con 0x7f3ec8102a60 2024-09-21T02:52:05.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 msgr2=0x7f3eb0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 0x7f3eb0042080 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f3ebc0096c0 tx=0x7f3ebc0058d0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 msgr2=0x7f3ec819c670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec819c670 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800e9f0 tx=0x7f3eb800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 shutdown_connections 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3eb003fb70 0x7f3eb0042080 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3ec81083a0 0x7f3ec819cc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3ec8108cf0 0x7f3ec81a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 --2- 172.21.15.16:0/1367315360 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ec8102a60 0x7f3ec819c670 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 >> 172.21.15.16:0/1367315360 conn(0x7f3ec8079730 msgr2=0x7f3ec810ffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 shutdown_connections 2024-09-21T02:52:05.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.308+0000 7f3ece270640 1 -- 172.21.15.16:0/1367315360 wait complete. 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3783471471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e0810de30 msgr2=0x7f8e08110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3783471471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e0810de30 0x7f8e08110320 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ca0 tx=0x7f8e0002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3783471471 shutdown_connections 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3783471471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e0810de30 0x7f8e08110320 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3783471471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e08105320 0x7f8e0810d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.984+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3783471471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e08104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.985+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3783471471 >> 172.21.15.16:0/3783471471 conn(0x7f8e08079740 msgr2=0x7f8e08079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.985+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3783471471 shutdown_connections 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.985+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3783471471 wait complete. 2024-09-21T02:52:05.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.985+0000 7f8e0c8e2640 1 Processor -- start 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.985+0000 7f8e0c8e2640 1 -- start start 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e0819a3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 0x7f8e0819fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08113970 con 0x7f8e08105320 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8e081137f0 con 0x7f8e081049d0 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0c8e2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8e08113af0 con 0x7f8e0810de30 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e05d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e05d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33054/0 (socket says 172.21.15.16:33054) 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e05d9b640 1 -- 172.21.15.16:0/3803163825 learned_addr learned my addr 172.21.15.16:0/3803163825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0659c640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e0819a3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 0x7f8e0819fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 -- 172.21.15.16:0/3803163825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 msgr2=0x7f8e0819a3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e0819a3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 -- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 msgr2=0x7f8e0819a940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e06d9d640 1 -- 172.21.15.16:0/3803163825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e081a0310 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e0659c640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e0819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.986+0000 7f8e05d9b640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8e06d9d640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 0x7f8e0819fbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e00031040 tx=0x7f8e00032fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00003b70 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f8e00003d10 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e081a05a0 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00030ac0 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.987+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e081a6d60 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.988+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 17) ==== 51915+0+0 (secure 0 0 0) 0x7f8e00030c60 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.988+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e08104de0 con 0x7f8e0810de30 2024-09-21T02:52:05.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.988+0000 7f8df77fe640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 0x7f8df0042080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.988+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f8e00004040 con 0x7f8e0810de30 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.988+0000 7f8e0659c640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 0x7f8df0042080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.989+0000 7f8e0659c640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 0x7f8df0042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e0806c620 tx=0x7f8df8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:04.994+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8e0002ee00 con 0x7f8e0810de30 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.191+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f8e08108f30 con 0x7f8e0810de30 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.196+0000 7f8df77fe640 1 -- 172.21.15.16:0/3803163825 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v19)=0 v19) ==== 106+0+0 (secure 0 0 0) 0x7f8e0003b410 con 0x7f8e0810de30 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 msgr2=0x7f8df0042080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 0x7f8df0042080 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f8e0806c620 tx=0x7f8df8005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 msgr2=0x7f8e0819fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 0x7f8e0819fbe0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f8e00031040 tx=0x7f8e00032fb0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 shutdown_connections 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8df003fb70 0x7f8df0042080 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.198+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8e08105320 0x7f8e0819a940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.199+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8e0810de30 0x7f8e0819fbe0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.199+0000 7f8e0c8e2640 1 --2- 172.21.15.16:0/3803163825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8e081049d0 0x7f8e0819a3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.199+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 >> 172.21.15.16:0/3803163825 conn(0x7f8e08079740 msgr2=0x7f8e08100840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.199+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 shutdown_connections 2024-09-21T02:52:05.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:05.199+0000 7f8e0c8e2640 1 -- 172.21.15.16:0/3803163825 wait complete. 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1407655403' entity='client.admin' 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: Standby manager daemon z started 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:52:06.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:06.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:52:06.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:05 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:06.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1407655403' entity='client.admin' 2024-09-21T02:52:06.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: Standby manager daemon z started 2024-09-21T02:52:06.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:52:06.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:06.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:52:06.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:05 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1407655403' entity='client.admin' 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: Standby manager daemon z started 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/crt"} : dispatch 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/z/key"} : dispatch 2024-09-21T02:52:06.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:05 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.123:0/149676667' entity='mgr.z' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 -- 172.21.15.16:0/694085365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 msgr2=0x7fb5e8104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/694085365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e8104cd0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb5d8009900 tx=0x7fb5d802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 -- 172.21.15.16:0/694085365 shutdown_connections 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/694085365 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/694085365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 0x7fb5e810d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/694085365 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e8104cd0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 -- 172.21.15.16:0/694085365 >> 172.21.15.16:0/694085365 conn(0x7fb5e8079760 msgr2=0x7fb5e8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 -- 172.21.15.16:0/694085365 shutdown_connections 2024-09-21T02:52:06.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.762+0000 7fb5ecd39640 1 -- 172.21.15.16:0/694085365 wait complete. 2024-09-21T02:52:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.763+0000 7fb5ecd39640 1 Processor -- start 2024-09-21T02:52:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.763+0000 7fb5ecd39640 1 -- start start 2024-09-21T02:52:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 0x7fb5e819a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e819fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8113990 con 0x7fb5e81048c0 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8113810 con 0x7fb5e810de70 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5ecd39640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e8113b10 con 0x7fb5e81052a0 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33096/0 (socket says 172.21.15.16:33096) 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e659c640 1 -- 172.21.15.16:0/158174257 learned_addr learned my addr 172.21.15.16:0/158174257 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e819fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e5d9b640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 0x7fb5e819a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 -- 172.21.15.16:0/158174257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 msgr2=0x7fb5e819a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 0x7fb5e819a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 -- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 msgr2=0x7fb5e819a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e6d9d640 1 -- 172.21.15.16:0/158174257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e81a0390 con 0x7fb5e810de70 2024-09-21T02:52:06.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.764+0000 7fb5e659c640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:06.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5e6d9d640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e819fc60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009fd0 tx=0x7fb5e000ef10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e000ce20 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fb5e0005020 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5e0010690 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e81a0680 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.765+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e81a6de0 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.766+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e806c190 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.767+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 18) ==== 103194+0+0 (secure 0 0 0) 0x7fb5e0004b20 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.767+0000 7fb5df7fe640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 0x7fb5d407d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.767+0000 7fb5e659c640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 0x7fb5d407d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.767+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fb5e009d4f0 con 0x7fb5e810de70 2024-09-21T02:52:06.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.768+0000 7fb5e659c640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 0x7fb5d407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80096c0 tx=0x7fb5d80058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:06.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.772+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb5e0004e10 con 0x7fb5e810de70 2024-09-21T02:52:06.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.962+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fb5e8108fc0 con 0x7fb5e810de70 2024-09-21T02:52:06.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.967+0000 7fb5df7fe640 1 -- 172.21.15.16:0/158174257 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v21)=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7fb5e0069e80 con 0x7fb5e810de70 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 msgr2=0x7fb5d407d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 0x7fb5d407d530 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5d80096c0 tx=0x7fb5d80058d0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 msgr2=0x7fb5e819fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e819fc60 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fb5e0009fd0 tx=0x7fb5e000ef10 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 shutdown_connections 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb5d407b020 0x7fb5d407d530 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb5e810de70 0x7fb5e819fc60 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb5e81052a0 0x7fb5e819a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 --2- 172.21.15.16:0/158174257 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb5e81048c0 0x7fb5e819a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:06.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 >> 172.21.15.16:0/158174257 conn(0x7fb5e8079760 msgr2=0x7fb5e8100960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 shutdown_connections 2024-09-21T02:52:06.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:06.969+0000 7fb5ecd39640 1 -- 172.21.15.16:0/158174257 wait complete. 2024-09-21T02:52:07.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:07 smithi016 ceph-mon[31253]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T02:52:07.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:52:07.660 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:07 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:52:07.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:07 smithi018 ceph-mon[34587]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T02:52:07.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:52:07.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:07 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:52:07.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:07 smithi123 ceph-mon[34645]: mgrmap e18: x(active, since 13s), standbys: z 2024-09-21T02:52:07.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "z", "id": "z"} : dispatch 2024-09-21T02:52:07.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:07 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:52:08.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:08 smithi016 ceph-mon[31253]: Standby manager daemon y started 2024-09-21T02:52:08.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:08 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:52:08.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:08 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:08.383 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:08 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:52:08.384 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:08 smithi016 ceph-mon[31253]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:08 smithi018 ceph-mon[34587]: Standby manager daemon y started 2024-09-21T02:52:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:08 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:52:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:08 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:08 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:52:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:08 smithi018 ceph-mon[34587]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:08 smithi123 ceph-mon[34645]: Standby manager daemon y started 2024-09-21T02:52:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:08 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/crt"} : dispatch 2024-09-21T02:52:08.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:08 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/crt"} : dispatch 2024-09-21T02:52:08.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:08 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/y/key"} : dispatch 2024-09-21T02:52:08.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:08 smithi123 ceph-mon[34645]: from='mgr.? 172.21.15.18:0/4151119370' entity='mgr.y' cmd={"prefix": "config-key get", "key": "mgr/dashboard/key"} : dispatch 2024-09-21T02:52:09.597 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:09 smithi016 ceph-mon[31253]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T02:52:09.598 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.616+0000 7fe634100640 1 -- 172.21.15.16:0/1825818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c1084c0 msgr2=0x7fe62c10ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.616+0000 7fe634100640 1 --2- 172.21.15.16:0/1825818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c1084c0 0x7fe62c10ecf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fe624009ca0 tx=0x7fe62402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.617+0000 7fe634100640 1 -- 172.21.15.16:0/1825818530 shutdown_connections 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.617+0000 7fe634100640 1 --2- 172.21.15.16:0/1825818530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c1084c0 0x7fe62c10ecf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.617+0000 7fe634100640 1 --2- 172.21.15.16:0/1825818530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c107b70 0x7fe62c107f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.617+0000 7fe634100640 1 --2- 172.21.15.16:0/1825818530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c102230 0x7fe62c102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.617+0000 7fe634100640 1 -- 172.21.15.16:0/1825818530 >> 172.21.15.16:0/1825818530 conn(0x7fe62c079e40 msgr2=0x7fe62c07a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.621+0000 7fe634100640 1 -- 172.21.15.16:0/1825818530 shutdown_connections 2024-09-21T02:52:09.622 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.621+0000 7fe634100640 1 -- 172.21.15.16:0/1825818530 wait complete. 2024-09-21T02:52:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.623 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.624 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.622+0000 7fe634100640 1 Processor -- start 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 -- start start 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c107b70 0x7fe62c197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c1084c0 0x7fe62c19c800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c1117c0 con 0x7fe62c107b70 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111640 con 0x7fe62c102230 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe634100640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe62c111940 con 0x7fe62c1084c0 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe631e9c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe631e9c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:50450/0 (socket says 172.21.15.16:50450) 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe631e9c640 1 -- 172.21.15.16:0/2403251743 learned_addr learned my addr 172.21.15.16:0/2403251743 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.623+0000 7fe63269d640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c1084c0 0x7fe62c19c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 -- 172.21.15.16:0/2403251743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c1084c0 msgr2=0x7fe62c19c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c1084c0 0x7fe62c19c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 -- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c107b70 msgr2=0x7fe62c197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c107b70 0x7fe62c197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 -- 172.21.15.16:0/2403251743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe62c19cee0 con 0x7fe62c102230 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.624+0000 7fe631e9c640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe62800e9f0 tx=0x7fe62800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.625+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe62800cd80 con 0x7fe62c102230 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.625+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(25 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe6280108c0 con 0x7fe62c102230 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.625+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe62c19d170 con 0x7fe62c102230 2024-09-21T02:52:09.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.625+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe628018820 con 0x7fe62c102230 2024-09-21T02:52:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.627+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe62c19d5e0 con 0x7fe62c102230 2024-09-21T02:52:09.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.630+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe62c102640 con 0x7fe62c102230 2024-09-21T02:52:09.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.631+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe628010430 con 0x7fe62c102230 2024-09-21T02:52:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.633+0000 7fe622ffd640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 0x7fe6180b8a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.633+0000 7fe63169b640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 0x7fe6180b8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:09.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.633+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe628010d50 con 0x7fe62c102230 2024-09-21T02:52:09.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.634+0000 7fe63169b640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 0x7fe6180b8a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0058c0 tx=0x7fe61c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:09.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.637+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6280adf80 con 0x7fe62c102230 2024-09-21T02:52:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:09 smithi018 ceph-mon[34587]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T02:52:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:52:09.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:09 smithi123 ceph-mon[34645]: mgrmap e19: x(active, since 15s), standbys: z, y 2024-09-21T02:52:09.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mgr metadata", "who": "y", "id": "y"} : dispatch 2024-09-21T02:52:09.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.838+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7fe62c10a2b0 con 0x7fe62c102230 2024-09-21T02:52:09.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.845+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 7 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6280ad390 con 0x7fe62c102230 2024-09-21T02:52:09.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.845+0000 7fe622ffd640 1 -- 172.21.15.16:0/2403251743 <== mon.2 v2:172.21.15.18:3300/0 8 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v22)=0 v22) ==== 123+0+0 (secure 0 0 0) 0x7fe6280b2360 con 0x7fe62c102230 2024-09-21T02:52:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 msgr2=0x7fe6180b8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:09.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 0x7fe6180b8a40 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe61c0058c0 tx=0x7fe61c005850 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 msgr2=0x7fe62c196fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe62800e9f0 tx=0x7fe62800eeb0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 shutdown_connections 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6180b6530 0x7fe6180b8a40 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe62c107b70 0x7fe62c197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe62c1084c0 0x7fe62c19c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 --2- 172.21.15.16:0/2403251743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe62c102230 0x7fe62c196fc0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 >> 172.21.15.16:0/2403251743 conn(0x7fe62c079e40 msgr2=0x7fe62c105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 shutdown_connections 2024-09-21T02:52:09.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:09.848+0000 7fe634100640 1 -- 172.21.15.16:0/2403251743 wait complete. 2024-09-21T02:52:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.661 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.664 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:10.975 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:10 smithi016 ceph-mon[31253]: from='client.? ' entity='client.admin' 2024-09-21T02:52:11.165 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:10 smithi018 ceph-mon[34587]: from='client.? ' entity='client.admin' 2024-09-21T02:52:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:10 smithi123 ceph-mon[34645]: from='client.? ' entity='client.admin' 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 -- 172.21.15.16:0/2194772610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8102a70 msgr2=0x7f4ef8102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/2194772610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8102a70 0x7f4ef8102e80 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8009900 tx=0x7f4ee802a3f0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 -- 172.21.15.16:0/2194772610 shutdown_connections 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/2194772610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef8108d00 0x7f4ef810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/2194772610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef81083b0 0x7f4ef81087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/2194772610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8102a70 0x7f4ef8102e80 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 -- 172.21.15.16:0/2194772610 >> 172.21.15.16:0/2194772610 conn(0x7f4ef8079710 msgr2=0x7f4ef8079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:11.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 -- 172.21.15.16:0/2194772610 shutdown_connections 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.361+0000 7f4eff7f3640 1 -- 172.21.15.16:0/2194772610 wait complete. 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 Processor -- start 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 -- start start 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef8102a70 0x7f4ef819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef81083b0 0x7f4ef819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8113ae0 con 0x7f4ef8108d00 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8113960 con 0x7f4ef81083b0 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.362+0000 7f4eff7f3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4ef8113c60 con 0x7f4ef8102a70 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33130/0 (socket says 172.21.15.16:33130) 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 -- 172.21.15.16:0/4254149786 learned_addr learned my addr 172.21.15.16:0/4254149786 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efd58f640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef8102a70 0x7f4ef819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:11.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efcd8e640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef81083b0 0x7f4ef819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 -- 172.21.15.16:0/4254149786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef8102a70 msgr2=0x7f4ef819a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef8102a70 0x7f4ef819a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 -- 172.21.15.16:0/4254149786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef81083b0 msgr2=0x7f4ef819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef81083b0 0x7f4ef819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 -- 172.21.15.16:0/4254149786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ef81a03a0 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4efdd90640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000cce0 tx=0x7f4ef000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef000f450 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef000faa0 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ef81a0690 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef0018b70 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.363+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ef81a6eb0 con 0x7f4ef8108d00 2024-09-21T02:52:11.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.364+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ef8103cf0 con 0x7f4ef8108d00 2024-09-21T02:52:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.366+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f4ef0018d10 con 0x7f4ef8108d00 2024-09-21T02:52:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.367+0000 7f4ef67fc640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 0x7f4ee00b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.367+0000 7f4efd58f640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 0x7f4ee00b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.367+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f4ef00e5bf0 con 0x7f4ef8108d00 2024-09-21T02:52:11.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.368+0000 7f4efd58f640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 0x7f4ee00b8a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8031440 tx=0x7f4ee8005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:11.371 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:11.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.370+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4ef0014030 con 0x7f4ef8108d00 2024-09-21T02:52:11.399 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.565+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=client_mount_timeout}] v 0) -- 0x7f4ef819b620 con 0x7f4ef8108d00 2024-09-21T02:52:11.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.570+0000 7f4ef67fc640 1 -- 172.21.15.16:0/4254149786 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=client_mount_timeout}]=0 v23)=0 v23) ==== 119+0+0 (secure 0 0 0) 0x7f4ef00ae190 con 0x7f4ef8108d00 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 msgr2=0x7f4ee00b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 0x7f4ee00b8a90 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4ee8031440 tx=0x7f4ee8005d50 comp rx=0 tx=0).stop 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 msgr2=0x7f4ef819fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f4ef000cce0 tx=0x7f4ef000a620 comp rx=0 tx=0).stop 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 shutdown_connections 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4ee00b6580 0x7f4ee00b8a90 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4ef8108d00 0x7f4ef819fd30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4ef81083b0 0x7f4ef819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 --2- 172.21.15.16:0/4254149786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ef8102a70 0x7f4ef819a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.572+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 >> 172.21.15.16:0/4254149786 conn(0x7f4ef8079710 msgr2=0x7f4ef8106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.573+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 shutdown_connections 2024-09-21T02:52:11.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:11.573+0000 7f4eff7f3640 1 -- 172.21.15.16:0/4254149786 wait complete. 2024-09-21T02:52:12.704 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:12 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4254149786' entity='client.admin' 2024-09-21T02:52:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 -- 172.21.15.16:0/74819886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc64109e30 msgr2=0x7fcc64110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/74819886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc64109e30 0x7fcc64110660 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c009ca0 tx=0x7fcc5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 -- 172.21.15.16:0/74819886 shutdown_connections 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/74819886 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc64109e30 0x7fcc64110660 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/74819886 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc641094e0 0x7fcc641098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/74819886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64103ba0 0x7fcc64103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 -- 172.21.15.16:0/74819886 >> 172.21.15.16:0/74819886 conn(0x7fcc640ff5d0 msgr2=0x7fcc64101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 -- 172.21.15.16:0/74819886 shutdown_connections 2024-09-21T02:52:12.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.717+0000 7fcc6b25e640 1 -- 172.21.15.16:0/74819886 wait complete. 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 Processor -- start 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 -- start start 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 0x7fcc6419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64109e30 0x7fcc6419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64113b40 con 0x7fcc641094e0 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcc641139c0 con 0x7fcc64103ba0 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc6b25e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc64113cc0 con 0x7fcc64109e30 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.718+0000 7fcc68ffa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 0x7fcc6419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33152/0 (socket says 172.21.15.16:33152) 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 -- 172.21.15.16:0/573759868 learned_addr learned my addr 172.21.15.16:0/573759868 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc68ffa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 0x7fcc6419a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:50478/0 (socket says 172.21.15.16:50478) 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc697fb640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64109e30 0x7fcc6419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 -- 172.21.15.16:0/573759868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64109e30 msgr2=0x7fcc6419fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64109e30 0x7fcc6419fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 -- 172.21.15.16:0/573759868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 msgr2=0x7fcc6419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 0x7fcc6419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 -- 172.21.15.16:0/573759868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc641a0400 con 0x7fcc641094e0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc63fff640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800d740 tx=0x7fcc5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58017070 con 0x7fcc641094e0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc641a06f0 con 0x7fcc641094e0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc5800de50 con 0x7fcc641094e0 2024-09-21T02:52:12.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.719+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800f620 con 0x7fcc641094e0 2024-09-21T02:52:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.720+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc641a6e50 con 0x7fcc641094e0 2024-09-21T02:52:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.721+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc64103fb0 con 0x7fcc641094e0 2024-09-21T02:52:12.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.722+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fcc5801a070 con 0x7fcc641094e0 2024-09-21T02:52:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.723+0000 7fcc61ffb640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 0x7fcc4c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.724+0000 7fcc68ffa640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 0x7fcc4c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.724+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcc580e50a0 con 0x7fcc641094e0 2024-09-21T02:52:12.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.724+0000 7fcc68ffa640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 0x7fcc4c0b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009bd0 tx=0x7fcc54005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:12.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.727+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc580ad5b0 con 0x7fcc641094e0 2024-09-21T02:52:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.921+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_mds_balancer}] v 0) -- 0x7fcc6419b5c0 con 0x7fcc641094e0 2024-09-21T02:52:12.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.925+0000 7fcc61ffb640 1 -- 172.21.15.16:0/573759868 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds_balancer}]=0 v24)=0 v24) ==== 117+0+0 (secure 0 0 0) 0x7fcc6419b5c0 con 0x7fcc641094e0 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 msgr2=0x7fcc4c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 0x7fcc4c0b8a60 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fcc54009bd0 tx=0x7fcc54005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 msgr2=0x7fcc6419aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800d740 tx=0x7fcc5800dc00 comp rx=0 tx=0).stop 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 shutdown_connections 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcc4c0b6550 0x7fcc4c0b8a60 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc641094e0 0x7fcc6419aa30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64109e30 0x7fcc6419fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 --2- 172.21.15.16:0/573759868 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc64103ba0 0x7fcc6419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 >> 172.21.15.16:0/573759868 conn(0x7fcc640ff5d0 msgr2=0x7fcc641069e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 shutdown_connections 2024-09-21T02:52:12.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:12.927+0000 7fcc6b25e640 1 -- 172.21.15.16:0/573759868 wait complete. 2024-09-21T02:52:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:12 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4254149786' entity='client.admin' 2024-09-21T02:52:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:12 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4254149786' entity='client.admin' 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 -- 172.21.15.16:0/1878385826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 msgr2=0x7f9a14102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 --2- 172.21.15.16:0/1878385826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a14102e80 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9a04009900 tx=0x7f9a0402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 -- 172.21.15.16:0/1878385826 shutdown_connections 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 --2- 172.21.15.16:0/1878385826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a14108d00 0x7f9a1410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 --2- 172.21.15.16:0/1878385826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a141083b0 0x7f9a141087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 --2- 172.21.15.16:0/1878385826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a14102e80 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 -- 172.21.15.16:0/1878385826 >> 172.21.15.16:0/1878385826 conn(0x7f9a14079710 msgr2=0x7f9a14079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:13.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 -- 172.21.15.16:0/1878385826 shutdown_connections 2024-09-21T02:52:13.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.882+0000 7f9a19477640 1 -- 172.21.15.16:0/1878385826 wait complete. 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.883+0000 7f9a19477640 1 Processor -- start 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.883+0000 7f9a19477640 1 -- start start 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a141083b0 0x7f9a1419ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a14108d00 0x7f9a1419fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113b80 con 0x7f9a14102a70 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113a00 con 0x7f9a14108d00 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a19477640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a14113d00 con 0x7f9a141083b0 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33174/0 (socket says 172.21.15.16:33174) 2024-09-21T02:52:13.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 -- 172.21.15.16:0/3613869478 learned_addr learned my addr 172.21.15.16:0/3613869478 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a137fe640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a14108d00 0x7f9a1419fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a0a7fc640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a141083b0 0x7f9a1419ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 -- 172.21.15.16:0/3613869478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a141083b0 msgr2=0x7f9a1419ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a141083b0 0x7f9a1419ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 -- 172.21.15.16:0/3613869478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a14108d00 msgr2=0x7f9a1419fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a14108d00 0x7f9a1419fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 -- 172.21.15.16:0/3613869478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a141a0560 con 0x7f9a14102a70 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a12ffd640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005ab0 tx=0x7f9a04003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.884+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a0403f070 con 0x7f9a14102a70 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.885+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a04031d60 con 0x7f9a14102a70 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.885+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a141a07f0 con 0x7f9a14102a70 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.885+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a04035030 con 0x7f9a14102a70 2024-09-21T02:52:13.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.885+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a141a6fb0 con 0x7f9a14102a70 2024-09-21T02:52:13.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.886+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a14103cf0 con 0x7f9a14102a70 2024-09-21T02:52:13.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.887+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f9a04042070 con 0x7f9a14102a70 2024-09-21T02:52:13.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.888+0000 7f9a10ff9640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 0x7f9a000b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.888+0000 7f9a0a7fc640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 0x7f9a000b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.888+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9a041062b0 con 0x7f9a14102a70 2024-09-21T02:52:13.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.889+0000 7f9a0a7fc640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 0x7f9a000b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a14108180 tx=0x7f99fc00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:13.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:13.891+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a040d2b20 con 0x7f9a14102a70 2024-09-21T02:52:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:13 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/573759868' entity='client.admin' 2024-09-21T02:52:13.965 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:13 smithi016 ceph-mon[31253]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:14.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.082+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_op_complaint_time}] v 0) -- 0x7f9a14102e80 con 0x7f9a14102a70 2024-09-21T02:52:14.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.086+0000 7f9a10ff9640 1 -- 172.21.15.16:0/3613869478 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_op_complaint_time}]=0 v25)=0 v25) ==== 121+0+0 (secure 0 0 0) 0x7f9a040d7a60 con 0x7f9a14102a70 2024-09-21T02:52:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 msgr2=0x7f9a000b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 0x7f9a000b89e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9a14108180 tx=0x7f99fc00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 msgr2=0x7f9a1419a5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f9a04005ab0 tx=0x7f9a04003d50 comp rx=0 tx=0).stop 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 shutdown_connections 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a000b64d0 0x7f9a000b89e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a14108d00 0x7f9a1419fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a141083b0 0x7f9a1419ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 --2- 172.21.15.16:0/3613869478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a14102a70 0x7f9a1419a5f0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 >> 172.21.15.16:0/3613869478 conn(0x7f9a14079710 msgr2=0x7f9a141064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 shutdown_connections 2024-09-21T02:52:14.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.089+0000 7f9a19477640 1 -- 172.21.15.16:0/3613869478 wait complete. 2024-09-21T02:52:14.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:13 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/573759868' entity='client.admin' 2024-09-21T02:52:14.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:13 smithi018 ceph-mon[34587]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:14.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:13 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/573759868' entity='client.admin' 2024-09-21T02:52:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:13 smithi123 ceph-mon[34645]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- 172.21.15.16:0/2699612355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698102650 msgr2=0x7f4698102a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2699612355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698102650 0x7f4698102a60 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f4694009900 tx=0x7f469402a480 comp rx=0 tx=0).stop 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- 172.21.15.16:0/2699612355 shutdown_connections 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2699612355 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698110200 0x7f4698112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2699612355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 0x7f469810fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2699612355 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698102650 0x7f4698102a60 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- 172.21.15.16:0/2699612355 >> 172.21.15.16:0/2699612355 conn(0x7f4698079770 msgr2=0x7f4698079b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- 172.21.15.16:0/2699612355 shutdown_connections 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- 172.21.15.16:0/2699612355 wait complete. 2024-09-21T02:52:14.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 Processor -- start 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.514+0000 7f46a00ab640 1 -- start start 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698102650 0x7f469819c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 0x7f469819cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4698115cf0 con 0x7f4698110200 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4698115b70 con 0x7f4698102650 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f46a00ab640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4698115e70 con 0x7f4698102fa0 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54468/0 (socket says 172.21.15.16:54468) 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 -- 172.21.15.16:0/2125598385 learned_addr learned my addr 172.21.15.16:0/2125598385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469de47640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698102650 0x7f469819c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469d646640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 0x7f469819cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 -- 172.21.15.16:0/2125598385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 msgr2=0x7f469819cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 0x7f469819cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 -- 172.21.15.16:0/2125598385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698102650 msgr2=0x7f469819c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698102650 0x7f469819c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 -- 172.21.15.16:0/2125598385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46981a2670 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.515+0000 7f469e648640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f469000cad0 tx=0x7f469000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.516+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469000ae20 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.516+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4690005020 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.516+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46981a2960 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.516+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f469000f6b0 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.516+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46981a90c0 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.517+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f469806c260 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.518+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f46900051d0 con 0x7f4698110200 2024-09-21T02:52:14.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.519+0000 7f468effd640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 0x7f46840b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.519+0000 7f469de47640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 0x7f46840b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.519+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f46900e5ad0 con 0x7f4698110200 2024-09-21T02:52:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.520+0000 7f469de47640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 0x7f46840b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4694031440 tx=0x7f4694009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:14.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.522+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4690014030 con 0x7f4698110200 2024-09-21T02:52:14.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.714+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_fragment_size_max}] v 0) -- 0x7f4698102a60 con 0x7f4698110200 2024-09-21T02:52:14.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.718+0000 7f468effd640 1 -- 172.21.15.16:0/2125598385 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_fragment_size_max}]=0 v26)=0 v26) ==== 127+0+0 (secure 0 0 0) 0x7f4698102a60 con 0x7f4698110200 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 msgr2=0x7f46840b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 0x7f46840b8a60 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f4694031440 tx=0x7f4694009560 comp rx=0 tx=0).stop 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 msgr2=0x7f46981a1f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f469000cad0 tx=0x7f469000cf90 comp rx=0 tx=0).stop 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 shutdown_connections 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46840b6550 0x7f46840b8a60 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4698110200 0x7f46981a1f40 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4698102fa0 0x7f469819cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 --2- 172.21.15.16:0/2125598385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4698102650 0x7f469819c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.720+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 >> 172.21.15.16:0/2125598385 conn(0x7f4698079770 msgr2=0x7f46981116e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.721+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 shutdown_connections 2024-09-21T02:52:14.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.721+0000 7f46a00ab640 1 -- 172.21.15.16:0/2125598385 wait complete. 2024-09-21T02:52:14.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 -- 172.21.15.16:0/1832209337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c501083b0 msgr2=0x7f9c501087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 --2- 172.21.15.16:0/1832209337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c501083b0 0x7f9c501087c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f9c44009900 tx=0x7f9c4402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 -- 172.21.15.16:0/1832209337 shutdown_connections 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 --2- 172.21.15.16:0/1832209337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c501083b0 0x7f9c501087c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 --2- 172.21.15.16:0/1832209337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 0x7f9c5010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 --2- 172.21.15.16:0/1832209337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c50102a70 0x7f9c50102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 -- 172.21.15.16:0/1832209337 >> 172.21.15.16:0/1832209337 conn(0x7f9c50079730 msgr2=0x7f9c50079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 -- 172.21.15.16:0/1832209337 shutdown_connections 2024-09-21T02:52:14.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.917+0000 7f9c54b25640 1 -- 172.21.15.16:0/1832209337 wait complete. 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 Processor -- start 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 -- start start 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c501083b0 0x7f9c50110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 0x7f9c50110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50113a90 con 0x7f9c50102a70 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50113910 con 0x7f9c501083b0 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c54b25640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9c50113c10 con 0x7f9c50108d00 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c4e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54494/0 (socket says 172.21.15.16:54494) 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c4e59c640 1 -- 172.21.15.16:0/3938825828 learned_addr learned my addr 172.21.15.16:0/3938825828 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:14.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.918+0000 7f9c4dd9b640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c501083b0 0x7f9c50110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4ed9d640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 0x7f9c50110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 -- 172.21.15.16:0/3938825828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 msgr2=0x7f9c50110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 0x7f9c50110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 -- 172.21.15.16:0/3938825828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c501083b0 msgr2=0x7f9c50110500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c501083b0 0x7f9c50110500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 -- 172.21.15.16:0/3938825828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c501a8730 con 0x7f9c50102a70 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c4e59c640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c920 tx=0x7f9c4000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40007c70 con 0x7f9c50102a70 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c501a8a20 con 0x7f9c50102a70 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9c40007e10 con 0x7f9c50102a70 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9c40018820 con 0x7f9c50102a70 2024-09-21T02:52:14.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.919+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c501a8f80 con 0x7f9c50102a70 2024-09-21T02:52:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.920+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c50103cf0 con 0x7f9c50102a70 2024-09-21T02:52:14.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.922+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f9c4000f450 con 0x7f9c50102a70 2024-09-21T02:52:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.922+0000 7f9c3f7fe640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 0x7f9c380b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.923+0000 7f9c4dd9b640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 0x7f9c380b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.923+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f9c400e5b50 con 0x7f9c50102a70 2024-09-21T02:52:14.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.923+0000 7f9c4dd9b640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 0x7f9c380b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:14.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:14.925+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9c40014030 con 0x7f9c50102a70 2024-09-21T02:52:15.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.117+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7f9c501112c0 con 0x7f9c50102a70 2024-09-21T02:52:15.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.120+0000 7f9c3f7fe640 1 -- 172.21.15.16:0/3938825828 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v27)=0 v27) ==== 133+0+0 (secure 0 0 0) 0x7f9c400ae070 con 0x7f9c50102a70 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 msgr2=0x7f9c380b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 0x7f9c380b89e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f9c44000c00 tx=0x7f9c44031400 comp rx=0 tx=0).stop 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 msgr2=0x7f9c5010ffc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f9c4000c920 tx=0x7f9c4000cde0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 shutdown_connections 2024-09-21T02:52:15.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9c380b64d0 0x7f9c380b89e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c50108d00 0x7f9c50110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9c501083b0 0x7f9c50110500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 --2- 172.21.15.16:0/3938825828 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9c50102a70 0x7f9c5010ffc0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 >> 172.21.15.16:0/3938825828 conn(0x7f9c50079730 msgr2=0x7f9c50105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 shutdown_connections 2024-09-21T02:52:15.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.123+0000 7f9c54b25640 1 -- 172.21.15.16:0/3938825828 wait complete. 2024-09-21T02:52:15.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3613869478' entity='client.admin' 2024-09-21T02:52:15.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2125598385' entity='client.admin' 2024-09-21T02:52:15.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:15 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3613869478' entity='client.admin' 2024-09-21T02:52:15.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:15 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2125598385' entity='client.admin' 2024-09-21T02:52:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:15 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3613869478' entity='client.admin' 2024-09-21T02:52:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:15 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2125598385' entity='client.admin' 2024-09-21T02:52:15.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 -- 172.21.15.16:0/1154416504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964410de10 msgr2=0x7f9644110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 --2- 172.21.15.16:0/1154416504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964410de10 0x7f9644110300 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f963c009ca0 tx=0x7f963c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 -- 172.21.15.16:0/1154416504 shutdown_connections 2024-09-21T02:52:15.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 --2- 172.21.15.16:0/1154416504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f964410de10 0x7f9644110300 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 --2- 172.21.15.16:0/1154416504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 0x7f964410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 --2- 172.21.15.16:0/1154416504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f96441049b0 0x7f9644104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 -- 172.21.15.16:0/1154416504 >> 172.21.15.16:0/1154416504 conn(0x7f9644079730 msgr2=0x7f9644079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 -- 172.21.15.16:0/1154416504 shutdown_connections 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.690+0000 7f964aeb8640 1 -- 172.21.15.16:0/1154416504 wait complete. 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 Processor -- start 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 -- start start 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 0x7f964419aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 0x7f964419fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113a90 con 0x7f96441049b0 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113910 con 0x7f9644105300 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f964aeb8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9644113c10 con 0x7f964410de10 2024-09-21T02:52:15.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f9648c54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f9648c54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54544/0 (socket says 172.21.15.16:54544) 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.691+0000 7f9648c54640 1 -- 172.21.15.16:0/3498926629 learned_addr learned my addr 172.21.15.16:0/3498926629 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9643fff640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 0x7f964419aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9649455640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 0x7f964419fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 -- 172.21.15.16:0/3498926629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 msgr2=0x7f964419fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 0x7f964419fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 -- 172.21.15.16:0/3498926629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 msgr2=0x7f964419aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 0x7f964419aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 -- 172.21.15.16:0/3498926629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96441a03d0 con 0x7f96441049b0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9649455640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 0x7f964419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9648c54640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f963800cd80 con 0x7f96441049b0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96441a06c0 con 0x7f96441049b0 2024-09-21T02:52:15.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f96380035f0 con 0x7f96441049b0 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.692+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9638010690 con 0x7f96441049b0 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.693+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96441a6e20 con 0x7f96441049b0 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.694+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9644104dc0 con 0x7f96441049b0 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.695+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f9638004000 con 0x7f96441049b0 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.696+0000 7f9641ffb640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 0x7f96340b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.696+0000 7f9643fff640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 0x7f96340b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.696+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f96380e5d10 con 0x7f96441049b0 2024-09-21T02:52:15.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.697+0000 7f9643fff640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 0x7f96340b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f964410fa50 tx=0x7f9630005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.701+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f96380ae1a0 con 0x7f96441049b0 2024-09-21T02:52:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 -- 172.21.15.16:0/2437005517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c1025d0 msgr2=0x7f1d6c1029e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 --2- 172.21.15.16:0/2437005517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c1029e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1d68009900 tx=0x7f1d6802a480 comp rx=0 tx=0).stop 2024-09-21T02:52:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 -- 172.21.15.16:0/2437005517 shutdown_connections 2024-09-21T02:52:15.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 --2- 172.21.15.16:0/2437005517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c110180 0x7f1d6c1125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 --2- 172.21.15.16:0/2437005517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c102f20 0x7f1d6c10fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 --2- 172.21.15.16:0/2437005517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c1029e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 -- 172.21.15.16:0/2437005517 >> 172.21.15.16:0/2437005517 conn(0x7f1d6c079730 msgr2=0x7f1d6c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 -- 172.21.15.16:0/2437005517 shutdown_connections 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.613+0000 7f1d73551640 1 -- 172.21.15.16:0/2437005517 wait complete. 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 Processor -- start 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 -- start start 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c19c660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c110180 0x7f1d6c1a1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c115c70 con 0x7f1d6c102f20 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c115af0 con 0x7f1d6c1025d0 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d73551640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1d6c115df0 con 0x7f1d6c110180 2024-09-21T02:52:15.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54522/0 (socket says 172.21.15.16:54522) 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 -- 172.21.15.16:0/1137691084 learned_addr learned my addr 172.21.15.16:0/1137691084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d712ed640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c19c660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d71aee640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c110180 0x7f1d6c1a1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 -- 172.21.15.16:0/1137691084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c110180 msgr2=0x7f1d6c1a1ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c110180 0x7f1d6c1a1ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 -- 172.21.15.16:0/1137691084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c1025d0 msgr2=0x7f1d6c19c660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c19c660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.614+0000 7f1d70aec640 1 -- 172.21.15.16:0/1137691084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d6c1a25d0 con 0x7f1d6c102f20 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d70aec640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00d950 tx=0x7f1d5c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c017030 con 0x7f1d6c102f20 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d5c004020 con 0x7f1d6c102f20 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d6c1a28c0 con 0x7f1d6c102f20 2024-09-21T02:52:15.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1d5c0047a0 con 0x7f1d6c102f20 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.615+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d6c1a9020 con 0x7f1d6c102f20 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.616+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d6c1029e0 con 0x7f1d6c102f20 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.618+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f1d5c009a30 con 0x7f1d6c102f20 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.619+0000 7f1d627fc640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 0x7f1d580b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.619+0000 7f1d712ed640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 0x7f1d580b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.619+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f1d5c0e5690 con 0x7f1d6c102f20 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.620+0000 7f1d712ed640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 0x7f1d580b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d68009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:15.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.623+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1d5c0adba0 con 0x7f1d6c102f20 2024-09-21T02:52:15.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.821+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_debug_frag}] v 0) -- 0x7f1d6c19d790 con 0x7f1d6c102f20 2024-09-21T02:52:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.825+0000 7f1d627fc640 1 -- 172.21.15.16:0/1137691084 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_debug_frag}]=0 v28)=0 v28) ==== 115+0+0 (secure 0 0 0) 0x7f1d5c0b1f80 con 0x7f1d6c102f20 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 msgr2=0x7f1d580b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 0x7f1d580b8a60 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1d68031440 tx=0x7f1d68009560 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 msgr2=0x7f1d6c19cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1d5c00d950 tx=0x7f1d5c00de10 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 shutdown_connections 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1d580b6550 0x7f1d580b8a60 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1d6c102f20 0x7f1d6c19cc00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1d6c110180 0x7f1d6c1a1ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 --2- 172.21.15.16:0/1137691084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1d6c1025d0 0x7f1d6c19c660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 >> 172.21.15.16:0/1137691084 conn(0x7f1d6c079730 msgr2=0x7f1d6c111600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 shutdown_connections 2024-09-21T02:52:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.827+0000 7f1d73551640 1 -- 172.21.15.16:0/1137691084 wait complete. 2024-09-21T02:52:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.902+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_ms}] v 0) -- 0x7f96441a7150 con 0x7f96441049b0 2024-09-21T02:52:15.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.902+0000 7f9641ffb640 1 -- 172.21.15.16:0/3498926629 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_ms}]=0 v28)=0 v28) ==== 106+0+0 (secure 0 0 0) 0x7f96380b2580 con 0x7f96441049b0 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 msgr2=0x7f96340b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 0x7f96340b89e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f964410fa50 tx=0x7f9630005f90 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 msgr2=0x7f964419a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f963800e9c0 tx=0x7f963800ee80 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 shutdown_connections 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f96340b64d0 0x7f96340b89e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f964410de10 0x7f964419fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9644105300 0x7f964419aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 --2- 172.21.15.16:0/3498926629 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f96441049b0 0x7f964419a460 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:15.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 >> 172.21.15.16:0/3498926629 conn(0x7f9644079730 msgr2=0x7f96440ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 shutdown_connections 2024-09-21T02:52:15.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:15.904+0000 7f964aeb8640 1 -- 172.21.15.16:0/3498926629 wait complete. 2024-09-21T02:52:16.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:16 smithi016 ceph-mon[31253]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:16.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3938825828' entity='client.admin' 2024-09-21T02:52:16.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1137691084' entity='client.admin' 2024-09-21T02:52:16.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:16 smithi018 ceph-mon[34587]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:16.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3938825828' entity='client.admin' 2024-09-21T02:52:16.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1137691084' entity='client.admin' 2024-09-21T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:16 smithi123 ceph-mon[34645]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3938825828' entity='client.admin' 2024-09-21T02:52:16.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1137691084' entity='client.admin' 2024-09-21T02:52:16.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 -- 172.21.15.16:0/325252847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 msgr2=0x7f91e0102380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 --2- 172.21.15.16:0/325252847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e0102380 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f91d0009900 tx=0x7f91d002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 -- 172.21.15.16:0/325252847 shutdown_connections 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 --2- 172.21.15.16:0/325252847 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 --2- 172.21.15.16:0/325252847 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 0x7f91e01098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 --2- 172.21.15.16:0/325252847 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e0102380 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 -- 172.21.15.16:0/325252847 >> 172.21.15.16:0/325252847 conn(0x7f91e0077c60 msgr2=0x7f91e00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 -- 172.21.15.16:0/325252847 shutdown_connections 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.764+0000 7f91e5702640 1 -- 172.21.15.16:0/325252847 wait complete. 2024-09-21T02:52:16.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.765+0000 7f91e5702640 1 Processor -- start 2024-09-21T02:52:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.765+0000 7f91e5702640 1 -- start start 2024-09-21T02:52:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 0x7f91e019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:16.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113ac0 con 0x7f91e0101f70 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113940 con 0x7f91e0109e00 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91e5702640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91e0113c40 con 0x7f91e01094b0 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54550/0 (socket says 172.21.15.16:54550) 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 -- 172.21.15.16:0/1066615457 learned_addr learned my addr 172.21.15.16:0/1066615457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91de7fc640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 0x7f91e019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:16.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91df7fe640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:16.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 -- 172.21.15.16:0/1066615457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 msgr2=0x7f91e019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:16.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 0x7f91e019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 -- 172.21.15.16:0/1066615457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 msgr2=0x7f91e019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91deffd640 1 -- 172.21.15.16:0/1066615457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91e01a0460 con 0x7f91e0101f70 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.766+0000 7f91df7fe640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.767+0000 7f91deffd640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005940 tx=0x7f91d0004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.767+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d003f030 con 0x7f91e0101f70 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.767+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f91d0003aa0 con 0x7f91e0101f70 2024-09-21T02:52:16.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.767+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91e01a06f0 con 0x7f91e0101f70 2024-09-21T02:52:16.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.767+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91d0035070 con 0x7f91e0101f70 2024-09-21T02:52:16.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.768+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91e01a6eb0 con 0x7f91e0101f70 2024-09-21T02:52:16.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.769+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91e0102380 con 0x7f91e0101f70 2024-09-21T02:52:16.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.770+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f91d0042030 con 0x7f91e0101f70 2024-09-21T02:52:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.770+0000 7f91c7fff640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 0x7f91c80b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.771+0000 7f91de7fc640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 0x7f91c80b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.771+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f91d0105d90 con 0x7f91e0101f70 2024-09-21T02:52:16.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.771+0000 7f91de7fc640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 0x7f91c80b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91e0109cf0 tx=0x7f91d400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:16.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.775+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f91d0109030 con 0x7f91e0101f70 2024-09-21T02:52:16.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.967+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/log_level}] v 0) -- 0x7f91e010bbd0 con 0x7f91e0101f70 2024-09-21T02:52:16.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.971+0000 7f91c7fff640 1 -- 172.21.15.16:0/1066615457 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/log_level}]=0 v29)=0 v29) ==== 129+0+0 (secure 0 0 0) 0x7f91d00d2600 con 0x7f91e0101f70 2024-09-21T02:52:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 msgr2=0x7f91c80b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 0x7f91c80b89e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f91e0109cf0 tx=0x7f91d400b390 comp rx=0 tx=0).stop 2024-09-21T02:52:16.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 msgr2=0x7f91e019a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f91d0005940 tx=0x7f91d0004270 comp rx=0 tx=0).stop 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 shutdown_connections 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f91c80b64d0 0x7f91c80b89e0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91e0109e00 0x7f91e019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.973+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91e01094b0 0x7f91e019aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.974+0000 7f91e5702640 1 --2- 172.21.15.16:0/1066615457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91e0101f70 0x7f91e019a4f0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:16.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.974+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 >> 172.21.15.16:0/1066615457 conn(0x7f91e0077c60 msgr2=0x7f91e01075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.974+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 shutdown_connections 2024-09-21T02:52:16.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:16.974+0000 7f91e5702640 1 -- 172.21.15.16:0/1066615457 wait complete. 2024-09-21T02:52:17.242 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'client', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T02:52:17.242 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:52:17.242 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_mds_balancer', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 -- 172.21.15.16:0/567244512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 msgr2=0x7f5fd010d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 --2- 172.21.15.16:0/567244512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd010d8b0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009900 tx=0x7f5fc402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 -- 172.21.15.16:0/567244512 shutdown_connections 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 --2- 172.21.15.16:0/567244512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd010d8b0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 --2- 172.21.15.16:0/567244512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 0x7f5fd0110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 --2- 172.21.15.16:0/567244512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 0x7f5fd0104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 -- 172.21.15.16:0/567244512 >> 172.21.15.16:0/567244512 conn(0x7f5fd0079730 msgr2=0x7f5fd0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:17.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 -- 172.21.15.16:0/567244512 shutdown_connections 2024-09-21T02:52:17.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.910+0000 7f5fd538c640 1 -- 172.21.15.16:0/567244512 wait complete. 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 Processor -- start 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 -- start start 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 0x7f5fd019a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 0x7f5fd019fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0113a70 con 0x7f5fd0105270 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd01138f0 con 0x7f5fd010ddf0 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fd538c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fd0113bf0 con 0x7f5fd0104840 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fce7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54562/0 (socket says 172.21.15.16:54562) 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fce7fc640 1 -- 172.21.15.16:0/3740731298 learned_addr learned my addr 172.21.15.16:0/3740731298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.911+0000 7f5fcf7fe640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 0x7f5fd019fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fceffd640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 0x7f5fd019a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 -- 172.21.15.16:0/3740731298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 msgr2=0x7f5fd019a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 0x7f5fd019a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 -- 172.21.15.16:0/3740731298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 msgr2=0x7f5fd019fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 0x7f5fd019fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 -- 172.21.15.16:0/3740731298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fd01a0410 con 0x7f5fd0105270 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fce7fc640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fc402a930 tx=0x7f5fc4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc403f030 con 0x7f5fd0105270 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5fc4003aa0 con 0x7f5fd0105270 2024-09-21T02:52:17.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5fc4035070 con 0x7f5fd0105270 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fd01a06a0 con 0x7f5fd0105270 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.912+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fd01a0b30 con 0x7f5fd0105270 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.914+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fd0104c50 con 0x7f5fd0105270 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.915+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f5fc4042030 con 0x7f5fd0105270 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.916+0000 7f5fb7fff640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 0x7f5fb80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:17.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.916+0000 7f5fceffd640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 0x7f5fb80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.916+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f5fc4105a80 con 0x7f5fd0105270 2024-09-21T02:52:17.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.917+0000 7f5fceffd640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 0x7f5fb80b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00058c0 tx=0x7f5fc000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:17.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:17.921+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5fd01a0b30 con 0x7f5fd0105270 2024-09-21T02:52:18.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.112+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_verify_scatter}] v 0) -- 0x7f5fd01a6ef0 con 0x7f5fd0105270 2024-09-21T02:52:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:17 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1066615457' entity='client.admin' 2024-09-21T02:52:18.114 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:17 smithi016 ceph-mon[31253]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:18.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.116+0000 7f5fb7fff640 1 -- 172.21.15.16:0/3740731298 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_verify_scatter}]=0 v30)=0 v30) ==== 119+0+0 (secure 0 0 0) 0x7f5fc40d2250 con 0x7f5fd0105270 2024-09-21T02:52:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 msgr2=0x7f5fb80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 0x7f5fb80b8a60 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f5fc00058c0 tx=0x7f5fc000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 msgr2=0x7f5fd019aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f5fc402a930 tx=0x7f5fc4004270 comp rx=0 tx=0).stop 2024-09-21T02:52:18.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 shutdown_connections 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5fb80b6550 0x7f5fb80b8a60 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fd0105270 0x7f5fd019aa40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fd010ddf0 0x7f5fd019fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 --2- 172.21.15.16:0/3740731298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fd0104840 0x7f5fd019a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.119+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 >> 172.21.15.16:0/3740731298 conn(0x7f5fd0079730 msgr2=0x7f5fd0106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.120+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 shutdown_connections 2024-09-21T02:52:18.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.120+0000 7f5fd538c640 1 -- 172.21.15.16:0/3740731298 wait complete. 2024-09-21T02:52:18.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:17 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1066615457' entity='client.admin' 2024-09-21T02:52:18.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:17 smithi018 ceph-mon[34587]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:18.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:17 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1066615457' entity='client.admin' 2024-09-21T02:52:18.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:17 smithi123 ceph-mon[34645]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:19.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 -- 172.21.15.16:0/3636164084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 msgr2=0x7f78cc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 --2- 172.21.15.16:0/3636164084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc110620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f78c4009ca0 tx=0x7f78c402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 -- 172.21.15.16:0/3636164084 shutdown_connections 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 --2- 172.21.15.16:0/3636164084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc110620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 --2- 172.21.15.16:0/3636164084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 0x7f78cc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 --2- 172.21.15.16:0/3636164084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 0x7f78cc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 -- 172.21.15.16:0/3636164084 >> 172.21.15.16:0/3636164084 conn(0x7f78cc07b850 msgr2=0x7f78cc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 -- 172.21.15.16:0/3636164084 shutdown_connections 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.998+0000 7f78d2a60640 1 -- 172.21.15.16:0/3636164084 wait complete. 2024-09-21T02:52:19.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 Processor -- start 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 -- start start 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 0x7f78cc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 0x7f78cc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc113ab0 con 0x7f78cc109df0 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc113930 con 0x7f78cc103b60 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d2a60640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f78cc113c30 con 0x7f78cc1094a0 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d0ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d0ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54566/0 (socket says 172.21.15.16:54566) 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78d0ffd640 1 -- 172.21.15.16:0/2100264559 learned_addr learned my addr 172.21.15.16:0/2100264559 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78cbfff640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 0x7f78cc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:18.999+0000 7f78c3fff640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 0x7f78cc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 -- 172.21.15.16:0/2100264559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 msgr2=0x7f78cc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 0x7f78cc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 -- 172.21.15.16:0/2100264559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 msgr2=0x7f78cc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 0x7f78cc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 -- 172.21.15.16:0/2100264559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78cc1a8720 con 0x7f78cc109df0 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d0ffd640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005cc0 tx=0x7f78c402f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c4003bf0 con 0x7f78cc109df0 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78cc1a89b0 con 0x7f78cc109df0 2024-09-21T02:52:19.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78c4003d90 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78c402fbc0 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.000+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78cc1a8f40 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.002+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f78ac005290 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.003+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f78c402fd60 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.004+0000 7f78c9ffb640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 0x7f78b80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.004+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f78c41068f0 con 0x7f78cc109df0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.005+0000 7f78cbfff640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 0x7f78b80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.005+0000 7f78cbfff640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 0x7f78b80b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78cc077160 tx=0x7f78bc00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:19.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.009+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f78c402f660 con 0x7f78cc109df0 2024-09-21T02:52:19.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.205+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7f78ac005580 con 0x7f78cc109df0 2024-09-21T02:52:19.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.209+0000 7f78c9ffb640 1 -- 172.21.15.16:0/2100264559 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v31)=0 v31) ==== 138+0+0 (secure 0 0 0) 0x7f78c40098b0 con 0x7f78cc109df0 2024-09-21T02:52:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 msgr2=0x7f78b80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 0x7f78b80b8a60 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f78cc077160 tx=0x7f78bc00b410 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 msgr2=0x7f78cc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f78c4005cc0 tx=0x7f78c402f380 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 shutdown_connections 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78b80b6550 0x7f78b80b8a60 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f78cc109df0 0x7f78cc076190 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f78cc1094a0 0x7f78cc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 --2- 172.21.15.16:0/2100264559 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f78cc103b60 0x7f78cc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 >> 172.21.15.16:0/2100264559 conn(0x7f78cc07b850 msgr2=0x7f78cc1074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 shutdown_connections 2024-09-21T02:52:19.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.212+0000 7f78d2a60640 1 -- 172.21.15.16:0/2100264559 wait complete. 2024-09-21T02:52:19.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 -- 172.21.15.16:0/1714617616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458108d20 msgr2=0x7f745810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 --2- 172.21.15.16:0/1714617616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458108d20 0x7f745810f550 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f7450009900 tx=0x7f745002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:19.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 -- 172.21.15.16:0/1714617616 shutdown_connections 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 --2- 172.21.15.16:0/1714617616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7458108d20 0x7f745810f550 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 --2- 172.21.15.16:0/1714617616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74581083d0 0x7f74581087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 --2- 172.21.15.16:0/1714617616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 0x7f7458102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 -- 172.21.15.16:0/1714617616 >> 172.21.15.16:0/1714617616 conn(0x7f7458079760 msgr2=0x7f7458079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 -- 172.21.15.16:0/1714617616 shutdown_connections 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.097+0000 7f745ef70640 1 -- 172.21.15.16:0/1714617616 wait complete. 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 Processor -- start 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 -- start start 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 0x7f745819a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 0x7f745819fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112990 con 0x7f74581083d0 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112810 con 0x7f7458108d20 2024-09-21T02:52:19.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.098+0000 7f745ef70640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7458112b10 con 0x7f7458102ae0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54572/0 (socket says 172.21.15.16:54572) 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 -- 172.21.15.16:0/1814365831 learned_addr learned my addr 172.21.15.16:0/1814365831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f745d50d640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 0x7f745819fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f745cd0c640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 0x7f745819a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 -- 172.21.15.16:0/1814365831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 msgr2=0x7f745819a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 0x7f745819a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 -- 172.21.15.16:0/1814365831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 msgr2=0x7f745819fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 0x7f745819fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 -- 172.21.15.16:0/1814365831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74581a0440 con 0x7f74581083d0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f745d50d640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 0x7f745819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7457fff640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f744800eb90 tx=0x7f744800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:19.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448010630 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74581a0730 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f744800ee90 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.099+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7448018ac0 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.100+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74581a6e90 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.101+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7458102ef0 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.102+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f7448018c60 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.103+0000 7f7455ffb640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 0x7f74440b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.103+0000 7f745cd0c640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 0x7f74440b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.103+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f74480e6180 con 0x7f74581083d0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.103+0000 7f745cd0c640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 0x7f74440b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7440005fd0 tx=0x7f744000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:19.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.106+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f74480b2950 con 0x7f74581083d0 2024-09-21T02:52:19.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.302+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mds_bal_split_size}] v 0) -- 0x7f745819b600 con 0x7f74581083d0 2024-09-21T02:52:19.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.306+0000 7f7455ffb640 1 -- 172.21.15.16:0/1814365831 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mds_bal_split_size}]=0 v32)=0 v32) ==== 118+0+0 (secure 0 0 0) 0x7f745819b600 con 0x7f74581083d0 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 msgr2=0x7f74440b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 0x7f74440b8a60 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f7440005fd0 tx=0x7f744000b390 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 msgr2=0x7f745819aa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f744800eb90 tx=0x7f744800c6a0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 shutdown_connections 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74440b6550 0x7f74440b8a60 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74581083d0 0x7f745819aa70 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7458108d20 0x7f745819fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 --2- 172.21.15.16:0/1814365831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7458102ae0 0x7f745819a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:19.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 >> 172.21.15.16:0/1814365831 conn(0x7f7458079760 msgr2=0x7f7458101b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 shutdown_connections 2024-09-21T02:52:19.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.309+0000 7f745ef70640 1 -- 172.21.15.16:0/1814365831 wait complete. 2024-09-21T02:52:19.425 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:19 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3740731298' entity='client.admin' 2024-09-21T02:52:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:19 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3740731298' entity='client.admin' 2024-09-21T02:52:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:19 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3740731298' entity='client.admin' 2024-09-21T02:52:20.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:20 smithi123 ceph-mon[34645]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:20.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2100264559' entity='client.admin' 2024-09-21T02:52:20.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1814365831' entity='client.admin' 2024-09-21T02:52:20.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3194752564' entity='client.admin' 2024-09-21T02:52:20.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:20 smithi016 ceph-mon[31253]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:20.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2100264559' entity='client.admin' 2024-09-21T02:52:20.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1814365831' entity='client.admin' 2024-09-21T02:52:20.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3194752564' entity='client.admin' 2024-09-21T02:52:20.594 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'debug_ms', '2'], check_status=True, hostname='smithi016') 2024-09-21T02:52:20.595 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_fragment_size_max', '10000'], check_status=True, hostname='smithi016') 2024-09-21T02:52:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:20 smithi018 ceph-mon[34587]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2100264559' entity='client.admin' 2024-09-21T02:52:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1814365831' entity='client.admin' 2024-09-21T02:52:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3194752564' entity='client.admin' 2024-09-21T02:52:21.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 -- 172.21.15.16:0/3035263473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 msgr2=0x7fef48103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/3035263473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef48103f70 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fef38009900 tx=0x7fef3802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 -- 172.21.15.16:0/3035263473 shutdown_connections 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/3035263473 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef48109df0 0x7fef48110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/3035263473 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef481094a0 0x7fef481098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/3035263473 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef48103f70 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 -- 172.21.15.16:0/3035263473 >> 172.21.15.16:0/3035263473 conn(0x7fef4807b850 msgr2=0x7fef4807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 -- 172.21.15.16:0/3035263473 shutdown_connections 2024-09-21T02:52:21.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.157+0000 7fef4cf1a640 1 -- 172.21.15.16:0/3035263473 wait complete. 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 Processor -- start 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 -- start start 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef481094a0 0x7fef4819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef48109df0 0x7fef4819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef48113ab0 con 0x7fef48103b60 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fef48113930 con 0x7fef481094a0 2024-09-21T02:52:21.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4cf1a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef48113c30 con 0x7fef48109df0 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54600/0 (socket says 172.21.15.16:54600) 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef4659c640 1 -- 172.21.15.16:0/2568743860 learned_addr learned my addr 172.21.15.16:0/2568743860 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef45d9b640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef481094a0 0x7fef4819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.158+0000 7fef46d9d640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef48109df0 0x7fef4819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 -- 172.21.15.16:0/2568743860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef48109df0 msgr2=0x7fef4819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef48109df0 0x7fef4819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 -- 172.21.15.16:0/2568743860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef481094a0 msgr2=0x7fef4819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef481094a0 0x7fef4819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 -- 172.21.15.16:0/2568743860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef481a0410 con 0x7fef48103b60 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4659c640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef38005940 tx=0x7fef380042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef3803f030 con 0x7fef48103b60 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef481a06a0 con 0x7fef48103b60 2024-09-21T02:52:21.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef38003cb0 con 0x7fef48103b60 2024-09-21T02:52:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef38035070 con 0x7fef48103b60 2024-09-21T02:52:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.159+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef481a6e60 con 0x7fef48103b60 2024-09-21T02:52:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.160+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef48104de0 con 0x7fef48103b60 2024-09-21T02:52:21.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.162+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fef38042030 con 0x7fef48103b60 2024-09-21T02:52:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.162+0000 7fef377fe640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 0x7fef300b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.163+0000 7fef45d9b640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 0x7fef300b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.163+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fef381061f0 con 0x7fef48103b60 2024-09-21T02:52:21.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.163+0000 7fef45d9b640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 0x7fef300b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef481091a0 tx=0x7fef3c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:21.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.166+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef380d2a60 con 0x7fef48103b60 2024-09-21T02:52:21.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.357+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_cephsqlite}] v 0) -- 0x7fef48103f70 con 0x7fef48103b60 2024-09-21T02:52:21.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.361+0000 7fef377fe640 1 -- 172.21.15.16:0/2568743860 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_cephsqlite}]=0 v34)=0 v34) ==== 115+0+0 (secure 0 0 0) 0x7fef48103f70 con 0x7fef48103b60 2024-09-21T02:52:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 msgr2=0x7fef300b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:21.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 0x7fef300b89e0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fef481091a0 tx=0x7fef3c00b390 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 msgr2=0x7fef4819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fef38005940 tx=0x7fef380042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 shutdown_connections 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef300b64d0 0x7fef300b89e0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef48109df0 0x7fef4819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef481094a0 0x7fef4819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 --2- 172.21.15.16:0/2568743860 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef48103b60 0x7fef4819a4a0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 >> 172.21.15.16:0/2568743860 conn(0x7fef4807b850 msgr2=0x7fef48107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 shutdown_connections 2024-09-21T02:52:21.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:21.364+0000 7fef4cf1a640 1 -- 172.21.15.16:0/2568743860 wait complete. 2024-09-21T02:52:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 -- 172.21.15.16:0/2823484628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914102ae0 msgr2=0x7f0914102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 --2- 172.21.15.16:0/2823484628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914102ae0 0x7f0914102ef0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f0910009900 tx=0x7f091002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 -- 172.21.15.16:0/2823484628 shutdown_connections 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 --2- 172.21.15.16:0/2823484628 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914108d70 0x7f091410f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 --2- 172.21.15.16:0/2823484628 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108420 0x7f0914108830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 --2- 172.21.15.16:0/2823484628 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914102ae0 0x7f0914102ef0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 -- 172.21.15.16:0/2823484628 >> 172.21.15.16:0/2823484628 conn(0x7f09140796b0 msgr2=0x7f0914079ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 -- 172.21.15.16:0/2823484628 shutdown_connections 2024-09-21T02:52:22.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.487+0000 7f091c3be640 1 -- 172.21.15.16:0/2823484628 wait complete. 2024-09-21T02:52:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.488+0000 7f091c3be640 1 Processor -- start 2024-09-21T02:52:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.488+0000 7f091c3be640 1 -- start start 2024-09-21T02:52:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.488+0000 7f091c3be640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914102ae0 0x7f091419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.488+0000 7f091c3be640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.488+0000 7f091c3be640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108d70 0x7f091419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091c3be640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0914112a80 con 0x7f0914108420 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091c3be640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0914112900 con 0x7f0914102ae0 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091c3be640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0914112c00 con 0x7f0914108d70 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54610/0 (socket says 172.21.15.16:54610) 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 -- 172.21.15.16:0/1687594812 learned_addr learned my addr 172.21.15.16:0/1687594812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091a15a640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914102ae0 0x7f091419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091a95b640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108d70 0x7f091419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 -- 172.21.15.16:0/1687594812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108d70 msgr2=0x7f091419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108d70 0x7f091419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 -- 172.21.15.16:0/1687594812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914102ae0 msgr2=0x7f091419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914102ae0 0x7f091419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 -- 172.21.15.16:0/1687594812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f09141a0470 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f0919959640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f090400d740 tx=0x7f090400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0904017070 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f090400de50 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f09141a0760 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.489+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f090400f620 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.490+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f09141a6ec0 con 0x7f0914108420 2024-09-21T02:52:22.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.490+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0914103d60 con 0x7f0914108420 2024-09-21T02:52:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.493+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f090401a070 con 0x7f0914108420 2024-09-21T02:52:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.494+0000 7f090b7fe640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 0x7f09000b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:22.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.494+0000 7f091a15a640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 0x7f09000b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.494+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f09040e5060 con 0x7f0914108420 2024-09-21T02:52:22.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.494+0000 7f091a15a640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 0x7f09000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0910005ab0 tx=0x7f0910005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:22.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.497+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f09040ad570 con 0x7f0914108420 2024-09-21T02:52:22.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:22 smithi016 ceph-mon[31253]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:22.604 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2568743860' entity='client.admin' 2024-09-21T02:52:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:22 smithi018 ceph-mon[34587]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2568743860' entity='client.admin' 2024-09-21T02:52:22.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.687+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=rados_mon_op_timeout}] v 0) -- 0x7f0914102ef0 con 0x7f0914108420 2024-09-21T02:52:22.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.691+0000 7f090b7fe640 1 -- 172.21.15.16:0/1687594812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=rados_mon_op_timeout}]=0 v35)=0 v35) ==== 120+0+0 (secure 0 0 0) 0x7f0914102ef0 con 0x7f0914108420 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 msgr2=0x7f09000b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 0x7f09000b8a60 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0910005ab0 tx=0x7f0910005a20 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 msgr2=0x7f091419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f090400d740 tx=0x7f090400dc00 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 shutdown_connections 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f09000b6550 0x7f09000b8a60 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0914108420 0x7f091419aaa0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0914108d70 0x7f091419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 --2- 172.21.15.16:0/1687594812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0914102ae0 0x7f091419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 >> 172.21.15.16:0/1687594812 conn(0x7f09140796b0 msgr2=0x7f0914101ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:22.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 shutdown_connections 2024-09-21T02:52:22.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:22.694+0000 7f091c3be640 1 -- 172.21.15.16:0/1687594812 wait complete. 2024-09-21T02:52:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:22 smithi123 ceph-mon[34645]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2568743860' entity='client.admin' 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3907956180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 msgr2=0x7fcd7810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3907956180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd7810f520 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fcd70009ca0 tx=0x7fcd7002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3907956180 shutdown_connections 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3907956180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd7810f520 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3907956180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd781083a0 0x7fcd781087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3907956180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd78102a60 0x7fcd78102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3907956180 >> 172.21.15.16:0/3907956180 conn(0x7fcd78079730 msgr2=0x7fcd78079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3907956180 shutdown_connections 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.969+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3907956180 wait complete. 2024-09-21T02:52:23.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.970+0000 7fcd7cf20640 1 Processor -- start 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.970+0000 7fcd7cf20640 1 -- start start 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 0x7fcd7810ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd781083a0 0x7fcd781104f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113a80 con 0x7fcd78108cf0 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113900 con 0x7fcd781083a0 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7cf20640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd78113c00 con 0x7fcd78102a60 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54586/0 (socket says 172.21.15.16:54586) 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 -- 172.21.15.16:0/3194752564 learned_addr learned my addr 172.21.15.16:0/3194752564 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd75d9b640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd781083a0 0x7fcd781104f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7659c640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 0x7fcd7810ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 -- 172.21.15.16:0/3194752564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 msgr2=0x7fcd7810ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 0x7fcd7810ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 -- 172.21.15.16:0/3194752564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd781083a0 msgr2=0x7fcd781104f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd781083a0 0x7fcd781104f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 -- 172.21.15.16:0/3194752564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd781a86d0 con 0x7fcd78108cf0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd7659c640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 0x7fcd7810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.971+0000 7fcd76d9d640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005cc0 tx=0x7fcd7002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.972+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd70003bf0 con 0x7fcd78108cf0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.972+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd70003d90 con 0x7fcd78108cf0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.972+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd781a8960 con 0x7fcd78108cf0 2024-09-21T02:52:23.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.972+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7002fbc0 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.972+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd781a8ea0 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.973+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcd78102e70 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.974+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fcd7002fd60 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.975+0000 7fcd677fe640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 0x7fcd600b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.975+0000 7fcd7659c640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 0x7fcd600b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.975+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fcd70106e60 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.975+0000 7fcd7659c640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 0x7fcd600b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005fd0 tx=0x7fcd68005ec0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:19.978+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcd700cf250 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.167+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=mon_op_complaint_time}] v 0) -- 0x7fcd781112b0 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.171+0000 7fcd677fe640 1 -- 172.21.15.16:0/3194752564 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_op_complaint_time}]=0 v33)=0 v33) ==== 121+0+0 (secure 0 0 0) 0x7fcd781112b0 con 0x7fcd78108cf0 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 msgr2=0x7fcd600b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 0x7fcd600b89e0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fcd68005fd0 tx=0x7fcd68005ec0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 msgr2=0x7fcd78110a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fcd70005cc0 tx=0x7fcd7002f380 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 shutdown_connections 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd600b64d0 0x7fcd600b89e0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd78108cf0 0x7fcd78110a30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd781083a0 0x7fcd781104f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 --2- 172.21.15.16:0/3194752564 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd78102a60 0x7fcd7810ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 >> 172.21.15.16:0/3194752564 conn(0x7fcd78079730 msgr2=0x7fcd78105240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 shutdown_connections 2024-09-21T02:52:23.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:20.173+0000 7fcd7cf20640 1 -- 172.21.15.16:0/3194752564 wait complete. 2024-09-21T02:52:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:23 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1687594812' entity='client.admin' 2024-09-21T02:52:24.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:23 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1687594812' entity='client.admin' 2024-09-21T02:52:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:23 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1687594812' entity='client.admin' 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2085333444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 msgr2=0x7fe1c010fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2085333444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c010fcc0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0009900 tx=0x7fe1b002a480 comp rx=0 tx=0).stop 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2085333444 shutdown_connections 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2085333444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c010fcc0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2085333444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0110200 0x7fe1c0112620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2085333444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0102650 0x7fe1c0102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2085333444 >> 172.21.15.16:0/2085333444 conn(0x7fe1c0079760 msgr2=0x7fe1c0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2085333444 shutdown_connections 2024-09-21T02:52:24.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.821+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2085333444 wait complete. 2024-09-21T02:52:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 Processor -- start 2024-09-21T02:52:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 -- start start 2024-09-21T02:52:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 0x7fe1c019c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:24.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0110200 0x7fe1c01a1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0115ca0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0115b20 con 0x7fe1c0102650 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c7d99640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1c0115e20 con 0x7fe1c0110200 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c5334640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c5334640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59878/0 (socket says 172.21.15.16:59878) 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c5334640 1 -- 172.21.15.16:0/2800461165 learned_addr learned my addr 172.21.15.16:0/2800461165 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.822+0000 7fe1c5b35640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 0x7fe1c019c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c6336640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0110200 0x7fe1c01a1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 -- 172.21.15.16:0/2800461165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0110200 msgr2=0x7fe1c01a1eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0110200 0x7fe1c01a1eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 -- 172.21.15.16:0/2800461165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 msgr2=0x7fe1c019c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:24.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 0x7fe1c019c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 -- 172.21.15.16:0/2800461165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1c01a2520 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5b35640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 0x7fe1c019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1c5334640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006010 tx=0x7fe1b00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b003f030 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.823+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1b0003800 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.824+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1c01a27b0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.824+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1b0035070 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.824+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1c01a2ce0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.825+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1c006c1f0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.826+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7fe1b0003cb0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.827+0000 7fe1b6ffd640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 0x7fe1ac0b8a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.827+0000 7fe1c5b35640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 0x7fe1ac0b8a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.827+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7fe1b0105d30 con 0x7fe1c0102fa0 2024-09-21T02:52:24.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.828+0000 7fe1c5b35640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 0x7fe1ac0b8a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0058c0 tx=0x7fe1bc00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:24.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:24.832+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe1c01a2ce0 con 0x7fe1c0102fa0 2024-09-21T02:52:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:24 smithi123 ceph-mon[34645]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:24 smithi016 ceph-mon[31253]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:25.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.019+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=osd_op_complaint_time}] v 0) -- 0x7fe1c01a91b0 con 0x7fe1c0102fa0 2024-09-21T02:52:25.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.023+0000 7fe1b6ffd640 1 -- 172.21.15.16:0/2800461165 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_op_complaint_time}]=0 v36)=0 v36) ==== 121+0+0 (secure 0 0 0) 0x7fe1b00d2500 con 0x7fe1c0102fa0 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 msgr2=0x7fe1ac0b8a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 0x7fe1ac0b8a90 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fe1bc0058c0 tx=0x7fe1bc00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 msgr2=0x7fe1c019cc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fe1b0006010 tx=0x7fe1b00042f0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 shutdown_connections 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1ac0b6580 0x7fe1ac0b8a90 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1c0102fa0 0x7fe1c019cc10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1c0110200 0x7fe1c01a1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 --2- 172.21.15.16:0/2800461165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe1c0102650 0x7fe1c019c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 >> 172.21.15.16:0/2800461165 conn(0x7fe1c0079760 msgr2=0x7fe1c0108c70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:25.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 shutdown_connections 2024-09-21T02:52:25.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:25.025+0000 7fe1c7d99640 1 -- 172.21.15.16:0/2800461165 wait complete. 2024-09-21T02:52:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:24 smithi018 ceph-mon[34587]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2800461165' entity='client.admin' 2024-09-21T02:52:26.141 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:26 smithi016 ceph-mon[31253]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:26 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2800461165' entity='client.admin' 2024-09-21T02:52:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:26 smithi018 ceph-mon[34587]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:26.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:26 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2800461165' entity='client.admin' 2024-09-21T02:52:26.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:26 smithi123 ceph-mon[34645]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 -- 172.21.15.16:0/2984594379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b41083d0 msgr2=0x7f16b41087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 --2- 172.21.15.16:0/2984594379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b41083d0 0x7f16b41087e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f16a0009900 tx=0x7f16a002a410 comp rx=0 tx=0).stop 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 -- 172.21.15.16:0/2984594379 shutdown_connections 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 --2- 172.21.15.16:0/2984594379 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b41083d0 0x7f16b41087e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 --2- 172.21.15.16:0/2984594379 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b4108d20 0x7f16b410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 --2- 172.21.15.16:0/2984594379 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4102ae0 0x7f16b4102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 -- 172.21.15.16:0/2984594379 >> 172.21.15.16:0/2984594379 conn(0x7f16b4079760 msgr2=0x7f16b4079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 -- 172.21.15.16:0/2984594379 shutdown_connections 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 -- 172.21.15.16:0/2984594379 wait complete. 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.211+0000 7f16bab77640 1 Processor -- start 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 -- start start 2024-09-21T02:52:27.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b41083d0 0x7f16b419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 0x7f16b419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4112990 con 0x7f16b4102ae0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4112810 con 0x7f16b4108d20 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16bab77640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f16b4112b10 con 0x7f16b41083d0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59910/0 (socket says 172.21.15.16:59910) 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 -- 172.21.15.16:0/1738875265 learned_addr learned my addr 172.21.15.16:0/1738875265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b3fff640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b41083d0 0x7f16b419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b9114640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 0x7f16b419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 -- 172.21.15.16:0/1738875265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b41083d0 msgr2=0x7f16b419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b41083d0 0x7f16b419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 -- 172.21.15.16:0/1738875265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 msgr2=0x7f16b419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 0x7f16b419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b8913640 1 -- 172.21.15.16:0/1738875265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16b41a04a0 con 0x7f16b4102ae0 2024-09-21T02:52:27.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.212+0000 7f16b9114640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 0x7f16b419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16b8913640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d950 tx=0x7f16a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a8017030 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16a8004020 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16b41a0790 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16a80047a0 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.213+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16b41a6ef0 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.214+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f16b4103d10 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.216+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f16a8009a30 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.216+0000 7f16b1ffb640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 0x7f16a40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.216+0000 7f16b3fff640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 0x7f16a40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.216+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f16a80e5880 con 0x7f16b4102ae0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.217+0000 7f16b3fff640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 0x7f16a40b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16a0000c00 tx=0x7f16a0031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:27.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.220+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f16a80add10 con 0x7f16b4102ae0 2024-09-21T02:52:27.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.411+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command([{prefix=config set, name=debug_client}] v 0) -- 0x7f16b4102ef0 con 0x7f16b4102ae0 2024-09-21T02:52:27.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.415+0000 7f16b1ffb640 1 -- 172.21.15.16:0/1738875265 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_client}]=0 v37)=0 v37) ==== 111+0+0 (secure 0 0 0) 0x7f16a80b20f0 con 0x7f16b4102ae0 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 msgr2=0x7f16a40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 0x7f16a40b89e0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f16a0000c00 tx=0x7f16a0031400 comp rx=0 tx=0).stop 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 msgr2=0x7f16b419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f16a800d950 tx=0x7f16a800de10 comp rx=0 tx=0).stop 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 shutdown_connections 2024-09-21T02:52:27.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f16a40b64d0 0x7f16a40b89e0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f16b4108d20 0x7f16b419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f16b41083d0 0x7f16b419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 --2- 172.21.15.16:0/1738875265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f16b4102ae0 0x7f16b419a530 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 >> 172.21.15.16:0/1738875265 conn(0x7f16b4079760 msgr2=0x7f16b406bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 shutdown_connections 2024-09-21T02:52:27.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:27.418+0000 7f16bab77640 1 -- 172.21.15.16:0/1738875265 wait complete. 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1738875265' entity='client.admin' 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:28.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:28 smithi016 ceph-mon[31253]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:52:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1738875265' entity='client.admin' 2024-09-21T02:52:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:52:28.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:28.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:28.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:28.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:28 smithi018 ceph-mon[34587]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:52:28.686 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_merge_size', '5'], check_status=True, hostname='smithi016') 2024-09-21T02:52:28.686 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_bits', '3'], check_status=True, hostname='smithi016') 2024-09-21T02:52:28.687 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_bal_split_size', '100'], check_status=True, hostname='smithi016') 2024-09-21T02:52:28.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1738875265' entity='client.admin' 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: Reconfiguring mon.b (monmap changed)... 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:28 smithi123 ceph-mon[34645]: Reconfiguring daemon mon.b on smithi018 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_frag', 'True'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_debug_scatterstat', 'True'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'mds_verify_scatter', 'True'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.086 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_mon_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mds', 'rados_osd_op_timeout', '900'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'client_mount_timeout', '30'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_cephsqlite', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_client', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_mgr', '20'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'debug_ms', '1'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/allow_m_granularity', 'True'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/dump_on_update', 'True'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.087 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mgr/snap_schedule/log_level', 'debug'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.088 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mgr', 'mon_warn_on_pool_no_app', 'False'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.088 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'mon', 'mon_op_complaint_time', '120'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.088 DEBUG:tasks.cephadm:waiting for RemoteProcess(client=, args=['sudo', '/home/ubuntu/cephtest/cephadm', '--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410', 'shell', '-c', '/etc/ceph/ceph.conf', '-k', '/etc/ceph/ceph.client.admin.keyring', '--fsid', 'ffc7910c-77c3-11ef-baf5-efdc52797490', '--', 'ceph', 'config', 'set', 'osd', 'osd_op_complaint_time', '180'], check_status=True, hostname='smithi016') 2024-09-21T02:52:29.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config dump 2024-09-21T02:52:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:29.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3412007272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 msgr2=0x7f77d010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3412007272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d010f530 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f77c8009900 tx=0x7f77c802a410 comp rx=0 tx=0).stop 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3412007272 shutdown_connections 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3412007272 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d010f530 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3412007272 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d01083b0 0x7f77d01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3412007272 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d0102a70 0x7f77d0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3412007272 >> 172.21.15.16:0/3412007272 conn(0x7f77d0079710 msgr2=0x7f77d0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3412007272 shutdown_connections 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.852+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3412007272 wait complete. 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 Processor -- start 2024-09-21T02:52:29.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 -- start start 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d0102a70 0x7f77d0199370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d01083b0 0x7f77d0199910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01129c0 con 0x7f77d0108d00 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0112840 con 0x7f77d0102a70 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.853+0000 7f77d4a2e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0112b40 con 0x7f77d01083b0 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59926/0 (socket says 172.21.15.16:59926) 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 -- 172.21.15.16:0/3918170691 learned_addr learned my addr 172.21.15.16:0/3918170691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77cdd9b640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d01083b0 0x7f77d0199910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ce59c640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d0102a70 0x7f77d0199370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 -- 172.21.15.16:0/3918170691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d01083b0 msgr2=0x7f77d0199910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d01083b0 0x7f77d0199910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:29.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 -- 172.21.15.16:0/3918170691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d0102a70 msgr2=0x7f77d0199370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d0102a70 0x7f77d0199370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 -- 172.21.15.16:0/3918170691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d019f2e0 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77ced9d640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f77c8004770 tx=0x7f77c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c803f030 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c8003aa0 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8035070 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d019f570 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.854+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a5d30 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.856+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f77d0103cf0 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.857+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f77c8042030 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.857+0000 7f77bf7fe640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 0x7f77b80b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.858+0000 7f77ce59c640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 0x7f77b80b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.858+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f77c8105d40 con 0x7f77d0108d00 2024-09-21T02:52:29.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.858+0000 7f77ce59c640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 0x7f77b80b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:29.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:29.862+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f77c80ce1d0 con 0x7f77d0108d00 2024-09-21T02:52:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.053+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config dump"} v 0) -- 0x7f77d010ab50 con 0x7f77d0108d00 2024-09-21T02:52:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.053+0000 7f77bf7fe640 1 -- 172.21.15.16:0/3918170691 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config dump"}]=0 v37) ==== 59+0+22127 (secure 0 0 0) 0x7f77c80d25b0 con 0x7f77d0108d00 2024-09-21T02:52:30.055 INFO:teuthology.orchestra.run.smithi016.stdout:WHO MASK LEVEL OPTION VALUE RO 2024-09-21T02:52:30.055 INFO:teuthology.orchestra.run.smithi016.stdout:global dev auth_debug true 2024-09-21T02:52:30.055 INFO:teuthology.orchestra.run.smithi016.stdout:global basic container_image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 * 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global dev debug_asserts_on_shutdown true 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_file true 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_journald false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global basic log_to_stderr false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_allow_pool_delete true 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_clock_drift_allowed 1.000000 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_cluster_log_to_file true 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_max_pg_per_osd 10000 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_pg_warn_max_object_skew 0.000000 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_crush_straw_calc_version_zero false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_legacy_crush_tunables false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_osd_down_out_interval_zero false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global dev mon_warn_on_pool_pg_num_not_power_of_two false 2024-09-21T02:52:30.056 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced mon_warn_on_too_few_osds false 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_bug true 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_old_message true 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:global dev ms_die_on_skipped_message false 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced osd_pool_default_erasure_code_profile plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:global advanced osd_pool_default_pg_autoscale_mode off 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_allow_insecure_global_id_reclaim false 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_mon_ticket_ttl 660.000000 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced auth_service_ticket_ttl 240.000000 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_mon 20/20 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_ms 1/1 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced debug_paxos 20/20 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_data_avail_warn 5 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_down_mkfs_grace 300 2024-09-21T02:52:30.057 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_mgr_mkfs_grace 240 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_op_complaint_time 120 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon dev mon_osd_prime_pg_temp true 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_osd_reporter_subtree_level osd 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim false 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced mon_warn_on_insecure_global_id_reclaim_allowed false 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mon advanced public_network 172.21.0.0/20 * 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced cephsqlite_lock_renewal_timeout 900000 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced client_mount_timeout 30 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_cephsqlite 20/20 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_client 20/20 2024-09-21T02:52:30.058 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_mgr 20/20 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced debug_ms 1/1 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/allow_ptrace true * 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/container_init True * 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/cephadm/migration_current 7 * 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/dashboard/ssl_server_port 8443 * 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/orchestrator/orchestrator cephadm 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/snap_schedule/allow_m_granularity true 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/snap_schedule/dump_on_update true 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mgr/snap_schedule/log_level debug 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mon_reweight_min_bytes_per_osd 10 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr advanced mon_reweight_min_pgs_per_osd 4 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:mgr dev mon_warn_on_pool_no_app false 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bdev_debug_aio true 2024-09-21T02:52:30.059 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced bdev_enable_discard true 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced bluestore_allocator bitmap * 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bluestore_block_size 96636764160 * 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev bluestore_fsck_on_mount true 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_bluefs 1/20 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_bluestore 1/20 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_ms 1/1 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_osd 20/20 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced debug_rocksdb 4/10 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_misdirected_ops true 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_op_order true 2024-09-21T02:52:30.060 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_pg_log_writeout true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_shutdown true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_cached_snaps true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_missing_on_start true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd dev osd_debug_verify_stray_on_activate true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_deep_scrub_update_digest_min_age 30 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_failsafe_full_ratio 0.950000 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_max_omap_entries_per_request 10 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_mclock_profile high_recovery_ops 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_memory_target_autotune true 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_objectstore bluestore * 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_complaint_time 180.000000 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_queue debug_random * 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_op_queue_cut_off debug_random * 2024-09-21T02:52:30.061 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_recover_clone_overlap true 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_recovery_max_chunk 1048576 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_scrub_load_threshold 5.000000 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_scrub_max_interval 600.000000 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:osd advanced osd_shutdown_pgref_assert true 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds 20/20 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_mds_balancer 20/20 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced debug_ms 2/2 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_fragment_size_max 10000 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_merge_size 5 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_split_bits 3 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_bal_split_size 100 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_frag true 2024-09-21T02:52:30.062 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_debug_scatterstat true 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced mds_op_complaint_time 180.000000 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:mds dev mds_verify_scatter true 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced osd_op_complaint_time 180.000000 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_mon_op_timeout 900 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:mds advanced rados_osd_op_timeout 900 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced client_mount_timeout 600 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_client 20/20 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced debug_ms 1/1 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_mon_op_timeout 900 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client advanced rados_osd_op_timeout 900 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_cache_enabled true 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_enable_ops_log true 2024-09-21T02:52:30.063 INFO:teuthology.orchestra.run.smithi016.stdout:client.rgw advanced rgw_enable_usage_log true 2024-09-21T02:52:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 msgr2=0x7f77b80b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 0x7f77b80b8a60 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f77c0005fd0 tx=0x7f77c0009500 comp rx=0 tx=0).stop 2024-09-21T02:52:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 msgr2=0x7f77d019ebb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f77c8004770 tx=0x7f77c8004270 comp rx=0 tx=0).stop 2024-09-21T02:52:30.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 shutdown_connections 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77b80b6550 0x7f77b80b8a60 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f77d0108d00 0x7f77d019ebb0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f77d01083b0 0x7f77d0199910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 --2- 172.21.15.16:0/3918170691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f77d0102a70 0x7f77d0199370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 >> 172.21.15.16:0/3918170691 conn(0x7f77d0079710 msgr2=0x7f77d01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 shutdown_connections 2024-09-21T02:52:30.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:30.056+0000 7f77d4a2e640 1 -- 172.21.15.16:0/3918170691 wait complete. 2024-09-21T02:52:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:52:30.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:30.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:30.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:30.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: Reconfiguring daemon mon.c on smithi123 2024-09-21T02:52:30.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3918170691' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:52:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: Reconfiguring daemon mon.c on smithi123 2024-09-21T02:52:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3918170691' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:52:30.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:30.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: Reconfiguring mon.c (monmap changed)... 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "mon."} : dispatch 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config get", "who": "mon", "key": "public_network"} : dispatch 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: Reconfiguring daemon mon.c on smithi123 2024-09-21T02:52:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3918170691' entity='client.admin' cmd={"prefix": "config dump"} : dispatch 2024-09-21T02:52:30.718 INFO:tasks.cephadm:Deploying OSDs... 2024-09-21T02:52:30.718 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:52:30.718 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:52:30.735 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:52:30.735 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_1 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 668 Links: 1 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 02:51:03.170604618 +0000 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 02:46:15.750148371 +0000 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 02:46:15.750148371 +0000 2024-09-21T02:52:30.791 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 02:46:15.750148371 +0000 2024-09-21T02:52:30.792 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:52:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T02:52:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T02:52:30.857 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000103859 s, 4.9 MB/s 2024-09-21T02:52:30.858 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:52:30.915 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_2 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 02:51:03.199604179 +0000 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 02:46:16.079143446 +0000 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 02:46:16.079143446 +0000 2024-09-21T02:52:30.971 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 02:46:16.079143446 +0000 2024-09-21T02:52:30.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:52:31.035 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T02:52:31.035 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T02:52:31.035 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000126815 s, 4.0 MB/s 2024-09-21T02:52:31.036 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:52:31.092 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_3 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 706 Links: 1 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:31.148 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 02:51:03.230603709 +0000 2024-09-21T02:52:31.149 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 02:46:16.414138432 +0000 2024-09-21T02:52:31.149 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 02:46:16.414138432 +0000 2024-09-21T02:52:31.149 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 02:46:16.414138432 +0000 2024-09-21T02:52:31.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:52:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T02:52:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T02:52:31.212 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000109649 s, 4.7 MB/s 2024-09-21T02:52:31.213 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:52:31.270 DEBUG:teuthology.orchestra.run.smithi016:> stat /dev/vg_nvme/lv_4 2024-09-21T02:52:31.325 INFO:teuthology.orchestra.run.smithi016.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 02:51:03.264603194 +0000 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 02:46:16.743133508 +0000 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 02:46:16.743133508 +0000 2024-09-21T02:52:31.326 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 02:46:16.743133508 +0000 2024-09-21T02:52:31.326 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:52:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records in 2024-09-21T02:52:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:1+0 records out 2024-09-21T02:52:31.390 INFO:teuthology.orchestra.run.smithi016.stderr:512 bytes copied, 0.000105354 s, 4.9 MB/s 2024-09-21T02:52:31.392 DEBUG:teuthology.orchestra.run.smithi016:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:52:31.448 DEBUG:teuthology.orchestra.run.smithi018:> set -ex 2024-09-21T02:52:31.449 DEBUG:teuthology.orchestra.run.smithi018:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:52:31.468 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:52:31.468 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_1 2024-09-21T02:52:31.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:31.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:31.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:31.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.526 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:52:31.526 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:31.526 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-21 02:52:02.073066831 +0000 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-21 02:46:32.131070097 +0000 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-21 02:46:32.131070097 +0000 2024-09-21T02:52:31.527 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-21 02:46:32.131070097 +0000 2024-09-21T02:52:31.527 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:52:31.596 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-21T02:52:31.596 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-21T02:52:31.596 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000107051 s, 4.8 MB/s 2024-09-21T02:52:31.597 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:52:31.657 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_2 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:31.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 684 Links: 1 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-21 02:52:02.074066820 +0000 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-21 02:46:32.436066488 +0000 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-21 02:46:32.436066488 +0000 2024-09-21T02:52:31.699 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-21 02:46:32.436066488 +0000 2024-09-21T02:52:31.700 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:52:31.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:31.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:31.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:31.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:31.765 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-21T02:52:31.766 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-21T02:52:31.766 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.0001207 s, 4.2 MB/s 2024-09-21T02:52:31.767 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:52:31.825 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_3 2024-09-21T02:52:31.881 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:52:31.881 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-21 02:52:02.074066820 +0000 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-21 02:46:32.745062831 +0000 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-21 02:46:32.745062831 +0000 2024-09-21T02:52:31.882 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-21 02:46:32.745062831 +0000 2024-09-21T02:52:31.882 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:52:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-21T02:52:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-21T02:52:31.947 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000102533 s, 5.0 MB/s 2024-09-21T02:52:31.948 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:52:32.005 DEBUG:teuthology.orchestra.run.smithi018:> stat /dev/vg_nvme/lv_4 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Access: 2024-09-21 02:52:02.075066808 +0000 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Modify: 2024-09-21 02:46:33.079058879 +0000 2024-09-21T02:52:32.061 INFO:teuthology.orchestra.run.smithi018.stdout:Change: 2024-09-21 02:46:33.079058879 +0000 2024-09-21T02:52:32.062 INFO:teuthology.orchestra.run.smithi018.stdout: Birth: 2024-09-21 02:46:33.079058879 +0000 2024-09-21T02:52:32.062 DEBUG:teuthology.orchestra.run.smithi018:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:52:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records in 2024-09-21T02:52:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:1+0 records out 2024-09-21T02:52:32.125 INFO:teuthology.orchestra.run.smithi018.stderr:512 bytes copied, 0.000115114 s, 4.4 MB/s 2024-09-21T02:52:32.127 DEBUG:teuthology.orchestra.run.smithi018:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:52:32.184 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2024-09-21T02:52:32.184 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2024-09-21T02:52:32.203 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-09-21T02:52:32.204 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:32.263 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 02:51:59.754664375 +0000 2024-09-21T02:52:32.264 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 02:46:32.758079754 +0000 2024-09-21T02:52:32.264 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 02:46:32.758079754 +0000 2024-09-21T02:52:32.264 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 02:46:32.758079754 +0000 2024-09-21T02:52:32.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-09-21T02:52:32.333 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T02:52:32.333 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T02:52:32.333 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000268267 s, 1.9 MB/s 2024-09-21T02:52:32.334 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-09-21T02:52:32.393 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 685 Links: 1 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 02:51:59.754664375 +0000 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 02:46:33.084071968 +0000 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 02:46:33.084071968 +0000 2024-09-21T02:52:32.454 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 02:46:33.084071968 +0000 2024-09-21T02:52:32.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-09-21T02:52:32.520 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T02:52:32.520 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T02:52:32.520 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000243791 s, 2.1 MB/s 2024-09-21T02:52:32.521 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-09-21T02:52:32.579 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2024-09-21T02:52:32.635 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-09-21T02:52:32.635 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:32.635 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-09-21T02:52:32.635 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:32.636 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:32.636 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 02:51:59.755664352 +0000 2024-09-21T02:52:32.636 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 02:46:33.403064348 +0000 2024-09-21T02:52:32.636 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 02:46:33.403064348 +0000 2024-09-21T02:52:32.636 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 02:46:33.403064348 +0000 2024-09-21T02:52:32.636 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-09-21T02:52:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:32 smithi018 ceph-mon[34587]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:32.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:32 smithi123 ceph-mon[34645]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T02:52:32.699 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T02:52:32.700 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000252623 s, 2.0 MB/s 2024-09-21T02:52:32.701 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-09-21T02:52:32.758 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2024-09-21T02:52:32.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:32 smithi016 ceph-mon[31253]: pgmap v12: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2024-09-21 02:51:59.755664352 +0000 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2024-09-21 02:46:33.726056632 +0000 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2024-09-21 02:46:33.726056632 +0000 2024-09-21T02:52:32.814 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: 2024-09-21 02:46:33.726056632 +0000 2024-09-21T02:52:32.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-09-21T02:52:32.877 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2024-09-21T02:52:32.877 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2024-09-21T02:52:32.877 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000249913 s, 2.0 MB/s 2024-09-21T02:52:32.878 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-09-21T02:52:32.935 INFO:tasks.cephadm:Deploying osd.0 on smithi016 with /dev/vg_nvme/lv_4... 2024-09-21T02:52:32.935 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:52:33.198 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:34.579 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:34 smithi016 ceph-mon[31253]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:34.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:34 smithi018 ceph-mon[34587]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:34.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:34 smithi123 ceph-mon[34645]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:35.649 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:52:35.677 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_4 2024-09-21T02:52:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 -- 172.21.15.16:0/3145737305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201094a0 msgr2=0x7f84201098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 --2- 172.21.15.16:0/3145737305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201094a0 0x7f84201098b0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f8414009900 tx=0x7f841402a410 comp rx=0 tx=0).stop 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 -- 172.21.15.16:0/3145737305 shutdown_connections 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 --2- 172.21.15.16:0/3145737305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f84201094a0 0x7f84201098b0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 --2- 172.21.15.16:0/3145737305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420109df0 0x7f8420110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 --2- 172.21.15.16:0/3145737305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8420103b60 0x7f8420103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 -- 172.21.15.16:0/3145737305 >> 172.21.15.16:0/3145737305 conn(0x7f842007b850 msgr2=0x7f842007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 -- 172.21.15.16:0/3145737305 shutdown_connections 2024-09-21T02:52:36.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.409+0000 7f8426588640 1 -- 172.21.15.16:0/3145737305 wait complete. 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 Processor -- start 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 -- start start 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201094a0 0x7f842019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 0x7f842019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8420113ab0 con 0x7f8420109df0 2024-09-21T02:52:36.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8420113930 con 0x7f8420103b60 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.410+0000 7f8426588640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8420113c30 con 0x7f84201094a0 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f8424b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 0x7f842019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:51436/0 (socket says 172.21.15.16:51436) 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f8424b25640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 0x7f842019fda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46902/0 (socket says 172.21.15.16:46902) 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201094a0 0x7f842019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 -- 172.21.15.16:0/3379713988 learned_addr learned my addr 172.21.15.16:0/3379713988 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 -- 172.21.15.16:0/3379713988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201094a0 msgr2=0x7f842019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201094a0 0x7f842019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 -- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 msgr2=0x7f842019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 0x7f842019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 -- 172.21.15.16:0/3379713988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84201a04d0 con 0x7f8420103b60 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841ffff640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:36.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8410017030 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8410004020 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84201a07c0 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84100047a0 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.411+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84201a6f20 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.412+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8420103f70 con 0x7f8420103b60 2024-09-21T02:52:36.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.414+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f8410009a30 con 0x7f8420103b60 2024-09-21T02:52:36.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.414+0000 7f841d7fa640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 0x7f84080b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:52:36.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.415+0000 7f841f7fe640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 0x7f84080b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:52:36.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.415+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1370+0+0 (secure 0 0 0) 0x7f84100e50f0 con 0x7f8420103b60 2024-09-21T02:52:36.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.415+0000 7f841f7fe640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 0x7f84080b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8414009230 tx=0x7f8414031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:52:36.419 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.418+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84100ad560 con 0x7f8420103b60 2024-09-21T02:52:36.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:36.616+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f8420063800 con 0x7f84080b6550 2024-09-21T02:52:36.618 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:36 smithi016 ceph-mon[31253]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:36.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:36 smithi018 ceph-mon[34587]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:36.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:36 smithi123 ceph-mon[34645]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:37.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:37 smithi016 ceph-mon[31253]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.528 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:52:37.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:52:37.529 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:37 smithi018 ceph-mon[34587]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:52:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:52:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:37.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:37 smithi123 ceph-mon[34645]: from='client.24296 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:52:37.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:52:37.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:52:37.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:38 smithi016 ceph-mon[31253]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:38.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:38 smithi018 ceph-mon[34587]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:38.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:38 smithi123 ceph-mon[34645]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:39.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:39 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"} : dispatch 2024-09-21T02:52:39.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:39 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"}]': finished 2024-09-21T02:52:39.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:39 smithi016 ceph-mon[31253]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:52:39.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:52:39.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:39 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"} : dispatch 2024-09-21T02:52:39.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:39 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"}]': finished 2024-09-21T02:52:39.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:39 smithi018 ceph-mon[34587]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:52:39.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:52:39.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:39 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"} : dispatch 2024-09-21T02:52:39.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:39 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1295059154' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c38a1c84-7e88-4346-bb53-088922f08bbe"}]': finished 2024-09-21T02:52:39.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:39 smithi123 ceph-mon[34645]: osdmap e6: 1 total, 0 up, 1 in 2024-09-21T02:52:39.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:52:40.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:40 smithi018 ceph-mon[34587]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:40.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:40 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/692083604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:52:40.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:40 smithi123 ceph-mon[34645]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:40.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:40 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/692083604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:52:40.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:40 smithi016 ceph-mon[31253]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:40.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:40 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/692083604' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:52:42.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:42 smithi018 ceph-mon[34587]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:42.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:42 smithi123 ceph-mon[34645]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:42.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:42 smithi016 ceph-mon[31253]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:44.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:44 smithi018 ceph-mon[34587]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:44.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:44 smithi123 ceph-mon[34645]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:44.754 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:44 smithi016 ceph-mon[31253]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:46.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:46 smithi018 ceph-mon[34587]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:46.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:46 smithi123 ceph-mon[34645]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:46.742 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:46 smithi016 ceph-mon[31253]: pgmap v20: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:52:47.643 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:47.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:52:47.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:47.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.0"} : dispatch 2024-09-21T02:52:47.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:48.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:48 smithi016 ceph-mon[31253]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:48.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:48 smithi016 ceph-mon[31253]: Deploying daemon osd.0 on smithi016 2024-09-21T02:52:48.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:48 smithi018 ceph-mon[34587]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:48.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:48 smithi018 ceph-mon[34587]: Deploying daemon osd.0 on smithi016 2024-09-21T02:52:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:48 smithi123 ceph-mon[34645]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:48 smithi123 ceph-mon[34645]: Deploying daemon osd.0 on smithi016 2024-09-21T02:52:50.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:50 smithi016 ceph-mon[31253]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:50.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:50 smithi018 ceph-mon[34587]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:50.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:50 smithi123 ceph-mon[34645]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:52.674 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:52 smithi016 ceph-mon[31253]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:52 smithi018 ceph-mon[34587]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:52.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:52 smithi123 ceph-mon[34645]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:54.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:54 smithi016 ceph-mon[31253]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:54.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:54.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:54.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:54 smithi018 ceph-mon[34587]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:54 smithi123 ceph-mon[34645]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:54.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:54.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:56 smithi018 ceph-mon[34587]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:56 smithi123 ceph-mon[34645]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:56 smithi016 ceph-mon[31253]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:56.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:56.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:57.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:57.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:57.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:57.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:57.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:57.692 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:57.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:52:57.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:52:57.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:57.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.737+0000 7f841d7fa640 1 -- 172.21.15.16:0/3379713988 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8420063800 con 0x7f84080b6550 2024-09-21T02:52:57.739 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 0 on host 'smithi016' 2024-09-21T02:52:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 msgr2=0x7f84080b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 0x7f84080b8a60 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f8414009230 tx=0x7f8414031400 comp rx=0 tx=0).stop 2024-09-21T02:52:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 msgr2=0x7f842019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:52:57.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f841000d950 tx=0x7f841000de10 comp rx=0 tx=0).stop 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 shutdown_connections 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84080b6550 0x7f84080b8a60 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8420109df0 0x7f842019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84201094a0 0x7f842019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 --2- 172.21.15.16:0/3379713988 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8420103b60 0x7f842019a560 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 >> 172.21.15.16:0/3379713988 conn(0x7f842007b850 msgr2=0x7f84201075f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 shutdown_connections 2024-09-21T02:52:57.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:52:57.740+0000 7f8426588640 1 -- 172.21.15.16:0/3379713988 wait complete. 2024-09-21T02:52:58.352 DEBUG:teuthology.orchestra.run.smithi016:osd.0> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.0.service 2024-09-21T02:52:58.356 INFO:tasks.cephadm:Deploying osd.1 on smithi018 with /dev/vg_nvme/lv_4... 2024-09-21T02:52:58.356 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:52:58.610 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:52:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:58 smithi016 ceph-mon[31253]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:58.883 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:52:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:58.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:58 smithi123 ceph-mon[34645]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:58.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:58.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:58.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:52:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:59.013 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:58 smithi018 ceph-mon[34587]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:52:59.013 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:52:59.013 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:52:59.013 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:52:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:00.266 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:52:59 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T02:52:59.994+0000 7f55abe3f640 -1 osd.0 0 log_to_monitors true 2024-09-21T02:53:00.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:00 smithi016 ceph-mon[31253]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:00.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:00 smithi016 ceph-mon[31253]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:53:00.922 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:00 smithi018 ceph-mon[34587]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:00.923 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:00 smithi018 ceph-mon[34587]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:53:00.925 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:53:00.950 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi018:vg_nvme/lv_4 2024-09-21T02:53:00.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:00 smithi123 ceph-mon[34645]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:00.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:00 smithi123 ceph-mon[34645]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]} : dispatch 2024-09-21T02:53:01.238 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:53:01.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 -- 172.21.15.18:0/1109903223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a30102a60 msgr2=0x7f9a30102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 --2- 172.21.15.18:0/1109903223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a30102a60 0x7f9a30102e70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c009900 tx=0x7f9a2c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:01.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 -- 172.21.15.18:0/1109903223 shutdown_connections 2024-09-21T02:53:01.681 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 --2- 172.21.15.18:0/1109903223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 0x7f9a3010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 --2- 172.21.15.18:0/1109903223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a301083a0 0x7f9a301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 --2- 172.21.15.18:0/1109903223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a30102a60 0x7f9a30102e70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 -- 172.21.15.18:0/1109903223 >> 172.21.15.18:0/1109903223 conn(0x7f9a30079720 msgr2=0x7f9a30079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 -- 172.21.15.18:0/1109903223 shutdown_connections 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 -- 172.21.15.18:0/1109903223 wait complete. 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.678+0000 7f9a38411640 1 Processor -- start 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 -- start start 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a30102a60 0x7f9a30104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 0x7f9a30104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30112960 con 0x7f9a30108cf0 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a301127e0 con 0x7f9a301083a0 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a38411640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a30112ae0 con 0x7f9a30102a60 2024-09-21T02:53:01.682 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:43048/0 (socket says 172.21.15.18:43048) 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a361ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a30102a60 0x7f9a30104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 -- 172.21.15.18:0/978365569 learned_addr learned my addr 172.21.15.18:0/978365569 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a369ae640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 0x7f9a30104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 -- 172.21.15.18:0/978365569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a30102a60 msgr2=0x7f9a30104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a30102a60 0x7f9a30104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 -- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 msgr2=0x7f9a30104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 0x7f9a30104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 -- 172.21.15.18:0/978365569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a301a8700 con 0x7f9a301083a0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.679+0000 7f9a359ac640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000d770 tx=0x7f9a2000dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.680+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a20017070 con 0x7f9a301083a0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.680+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a20009c30 con 0x7f9a301083a0 2024-09-21T02:53:01.683 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.680+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a301a89f0 con 0x7f9a301083a0 2024-09-21T02:53:01.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.680+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a2000f620 con 0x7f9a301083a0 2024-09-21T02:53:01.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.680+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a301a8f30 con 0x7f9a301083a0 2024-09-21T02:53:01.684 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.681+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a30103ce0 con 0x7f9a301083a0 2024-09-21T02:53:01.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.682+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f9a2001a070 con 0x7f9a301083a0 2024-09-21T02:53:01.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.683+0000 7f9a277fe640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 0x7f9a1c0b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:01.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.683+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(7..7 src has 1..7) ==== 1576+0+0 (secure 0 0 0) 0x7f9a200e5270 con 0x7f9a301083a0 2024-09-21T02:53:01.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.683+0000 7f9a361ad640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 0x7f9a1c0b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:01.685 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.684+0000 7f9a361ad640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 0x7f9a1c0b89e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000c00 tx=0x7f9a2c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:01.688 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.687+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9a20009de0 con 0x7f9a301083a0 2024-09-21T02:53:01.888 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:01.887+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f9a30106320 con 0x7f9a1c0b64d0 2024-09-21T02:53:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:53:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:53:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:53:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:01 smithi018 ceph-mon[34587]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:53:01.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:53:01.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:01 smithi123 ceph-mon[34645]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:53:02.015 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T02:53:01.703+0000 7f55a7dc1640 -1 osd.0 0 waiting for initial osdmap 2024-09-21T02:53:02.016 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T02:53:01.710+0000 7f55a3bbf640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: osdmap e7: 1 total, 0 up, 1 in 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd={"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:02.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:02.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:02.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:02.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:01 smithi016 ceph-mon[31253]: from='osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:53:02.822 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:02.822 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi016 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:02.823 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi016 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:02.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:03.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:03.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi016 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi016 to 17092M 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: osdmap e8: 1 total, 0 up, 1 in 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:03.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:03.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] boot 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:53:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] boot 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:53:04.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: from='client.24317 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: osd.0 [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] boot 2024-09-21T02:53:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: osdmap e9: 1 total, 1 up, 1 in 2024-09-21T02:53:04.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 0} : dispatch 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/2247572287' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"}]': finished 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:53:04.743 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/2247572287' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"}]': finished 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:53:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:05.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: pgmap v32: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: osdmap e10: 1 total, 1 up, 1 in 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/2247572287' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"} : dispatch 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ab60e425-6410-41cb-a003-2dc7e21c8cb0"}]': finished 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: osdmap e11: 2 total, 1 up, 2 in 2024-09-21T02:53:05.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:05.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/4242783894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:06.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/4242783894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:06.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/4242783894' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:07.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:06 smithi016 ceph-mon[31253]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:06 smithi018 ceph-mon[34587]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:06 smithi123 ceph-mon[34645]: pgmap v35: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:08.988 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:08 smithi018 ceph-mon[34587]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:09.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:08 smithi016 ceph-mon[31253]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:09.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:08 smithi123 ceph-mon[34645]: pgmap v36: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:10.831 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:10 smithi018 ceph-mon[34587]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:11.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:10 smithi016 ceph-mon[31253]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:10 smithi123 ceph-mon[34645]: pgmap v37: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:13.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:12 smithi016 ceph-mon[31253]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:13.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:12 smithi018 ceph-mon[34587]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:13.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:12 smithi123 ceph-mon[34645]: pgmap v38: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:13.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:53:13.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:14.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:53:14.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:14.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.1"} : dispatch 2024-09-21T02:53:14.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:14.853 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:14 smithi018 ceph-mon[34587]: Deploying daemon osd.1 on smithi018 2024-09-21T02:53:14.853 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:14 smithi018 ceph-mon[34587]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:15.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:14 smithi016 ceph-mon[31253]: Deploying daemon osd.1 on smithi018 2024-09-21T02:53:15.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:14 smithi016 ceph-mon[31253]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:14 smithi123 ceph-mon[34645]: Deploying daemon osd.1 on smithi018 2024-09-21T02:53:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:14 smithi123 ceph-mon[34645]: pgmap v39: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:16 smithi016 ceph-mon[31253]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:16 smithi018 ceph-mon[34587]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:16 smithi123 ceph-mon[34645]: pgmap v40: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:18.830 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:18 smithi018 ceph-mon[34587]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:19.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:18 smithi016 ceph-mon[31253]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:18 smithi123 ceph-mon[34645]: pgmap v41: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:19.981 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:19.981 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:19.981 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:20.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:20.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:20.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:20.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:21.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:20 smithi016 ceph-mon[31253]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:21.022 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:20 smithi018 ceph-mon[34587]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:20 smithi123 ceph-mon[34645]: pgmap v42: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:22.886 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:22.886 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:22.886 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:22.887 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:22.887 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:22.887 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:22.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:22.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:22.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:22.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: pgmap v43: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.507 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.506+0000 7f9a277fe640 1 -- 172.21.15.18:0/978365569 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f9a30106320 con 0x7f9a1c0b64d0 2024-09-21T02:53:23.508 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 1 on host 'smithi018' 2024-09-21T02:53:23.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 msgr2=0x7f9a1c0b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:23.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 0x7f9a1c0b89e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f9a2c000c00 tx=0x7f9a2c031400 comp rx=0 tx=0).stop 2024-09-21T02:53:23.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 msgr2=0x7f9a30104790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:23.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f9a2000d770 tx=0x7f9a2000dc30 comp rx=0 tx=0).stop 2024-09-21T02:53:23.510 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 shutdown_connections 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a1c0b64d0 0x7f9a1c0b89e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a30108cf0 0x7f9a30104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a301083a0 0x7f9a30104790 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 --2- 172.21.15.18:0/978365569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a30102a60 0x7f9a30104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 >> 172.21.15.18:0/978365569 conn(0x7f9a30079720 msgr2=0x7f9a30101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 shutdown_connections 2024-09-21T02:53:23.511 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:53:23.509+0000 7f9a38411640 1 -- 172.21.15.18:0/978365569 wait complete. 2024-09-21T02:53:23.809 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:23.809 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.809 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:23.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.917 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:24.120 DEBUG:teuthology.orchestra.run.smithi018:osd.1> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.1.service 2024-09-21T02:53:24.132 INFO:tasks.cephadm:Deploying osd.2 on smithi123 with /dev/vg_nvme/lv_4... 2024-09-21T02:53:24.132 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_4 2024-09-21T02:53:24.387 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:53:24.799 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:24 smithi123 ceph-mon[34645]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:24.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:24 smithi018 ceph-mon[34587]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:24.932 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:24 smithi016 ceph-mon[31253]: pgmap v44: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:25.432 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T02:53:25.379+0000 7f441ad1e640 -1 osd.1 0 log_to_monitors true 2024-09-21T02:53:25.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:25 smithi123 ceph-mon[34645]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:25.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:25 smithi123 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:25.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:25 smithi018 ceph-mon[34587]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:25.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:25 smithi018 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:26.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:25 smithi016 ceph-mon[31253]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:26.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:25 smithi016 ceph-mon[31253]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]} : dispatch 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:26.715 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:26.715 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T02:53:26.619+0000 7f4416ca0640 -1 osd.1 0 waiting for initial osdmap 2024-09-21T02:53:26.715 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T02:53:26.624+0000 7f4412a9e640 -1 osd.1 13 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:53:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:53:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:26.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:53:26.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:26.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:26 smithi123 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:26.767 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:53:26.793 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_4 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: pgmap v45: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: from='osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733]' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: osdmap e12: 2 total, 1 up, 2 in 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:26 smithi016 ceph-mon[31253]: from='osd.1 ' entity='osd.1' cmd={"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:53:27.041 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:53:27.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 -- 172.21.15.123:0/800915195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 msgr2=0x7f50fc10f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:27.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 --2- 172.21.15.123:0/800915195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc10f540 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f50f4009ca0 tx=0x7f50f402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 -- 172.21.15.123:0/800915195 shutdown_connections 2024-09-21T02:53:27.566 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 --2- 172.21.15.123:0/800915195 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc10f540 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 --2- 172.21.15.123:0/800915195 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc1083c0 0x7f50fc1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 --2- 172.21.15.123:0/800915195 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc102a80 0x7f50fc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 -- 172.21.15.123:0/800915195 >> 172.21.15.123:0/800915195 conn(0x7f50fc079740 msgr2=0x7f50fc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 -- 172.21.15.123:0/800915195 shutdown_connections 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.562+0000 7f5101f1d640 1 -- 172.21.15.123:0/800915195 wait complete. 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.563+0000 7f5101f1d640 1 Processor -- start 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.563+0000 7f5101f1d640 1 -- start start 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc102a80 0x7f50fc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc1083c0 0x7f50fc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113aa0 con 0x7f50fc1083c0 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113920 con 0x7f50fc102a80 2024-09-21T02:53:27.567 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f5101f1d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50fc113c20 con 0x7f50fc108d10 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50422/0 (socket says 172.21.15.123:50422) 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 -- 172.21.15.123:0/2985857740 learned_addr learned my addr 172.21.15.123:0/2985857740 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50faffd640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc1083c0 0x7f50fc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fb7fe640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc102a80 0x7f50fc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 -- 172.21.15.123:0/2985857740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc102a80 msgr2=0x7f50fc19a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc102a80 0x7f50fc19a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 -- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc1083c0 msgr2=0x7f50fc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc1083c0 0x7f50fc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 -- 172.21.15.123:0/2985857740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50fc1a0380 con 0x7f50fc108d10 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.564+0000 7f50fbfff640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005cc0 tx=0x7f50f402f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.565+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f4003bf0 con 0x7f50fc108d10 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.565+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50f4003d90 con 0x7f50fc108d10 2024-09-21T02:53:27.568 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.565+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50f402fbc0 con 0x7f50fc108d10 2024-09-21T02:53:27.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.565+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50fc1a0610 con 0x7f50fc108d10 2024-09-21T02:53:27.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.565+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50fc1a6dd0 con 0x7f50fc108d10 2024-09-21T02:53:27.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.566+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50fc103d00 con 0x7f50fc108d10 2024-09-21T02:53:27.569 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.567+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f50f402fd60 con 0x7f50fc108d10 2024-09-21T02:53:27.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.568+0000 7f50f8ff9640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 0x7f50e40b89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:27.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.568+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 2244+0+0 (secure 0 0 0) 0x7f50f4107140 con 0x7f50fc108d10 2024-09-21T02:53:27.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.568+0000 7f50fb7fe640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 0x7f50e40b89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:27.570 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.569+0000 7f50fb7fe640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 0x7f50e40b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005fd0 tx=0x7f50ec005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:27.573 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.572+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f50f40cf260 con 0x7f50fc108d10 2024-09-21T02:53:27.766 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:27.764+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f50fc10ab30 con 0x7f50e40b64d0 2024-09-21T02:53:27.766 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:53:27.766 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:53:27.766 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:27.766 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:27.766 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi018 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi018 to 17092M 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:27.767 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:27 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi018 2024-09-21T02:53:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi018 to 17092M 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:27.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:27 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:28.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: osdmap e13: 2 total, 1 up, 2 in 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi018 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi018 to 17092M 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:27 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] boot 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:28.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:28 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] boot 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:53:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:28.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:28.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:28.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:28.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:28 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: osd.1 [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] boot 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: osdmap e14: 2 total, 2 up, 2 in 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 1} : dispatch 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: from='client.24337 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:29.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:28 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:29.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:29 smithi123 ceph-mon[34645]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:53:30.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:29 smithi018 ceph-mon[34587]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:53:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:29 smithi016 ceph-mon[31253]: osdmap e15: 2 total, 2 up, 2 in 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/2418113951' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"}]': finished 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/3313346164' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/2418113951' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"}]': finished 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:31.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/3313346164' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: pgmap v51: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/2418113951' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"} : dispatch 2024-09-21T02:53:31.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "16bdfeb8-142c-4773-915d-037f7d9f90f7"}]': finished 2024-09-21T02:53:31.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: osdmap e16: 3 total, 2 up, 3 in 2024-09-21T02:53:31.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:31.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/3313346164' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:32 smithi123 ceph-mon[34645]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:32 smithi018 ceph-mon[34587]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:33.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:32 smithi016 ceph-mon[31253]: pgmap v53: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:35.060 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:34 smithi123 ceph-mon[34645]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:34 smithi018 ceph-mon[34587]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:35.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:34 smithi016 ceph-mon[31253]: pgmap v54: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:36 smithi123 ceph-mon[34645]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:36 smithi018 ceph-mon[34587]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:37.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:36 smithi016 ceph-mon[31253]: pgmap v55: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:39.057 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:38 smithi123 ceph-mon[34645]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:39.057 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:53:39.057 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:38 smithi018 ceph-mon[34587]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:53:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:39.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:38 smithi016 ceph-mon[31253]: pgmap v56: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:39.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.2"} : dispatch 2024-09-21T02:53:39.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:39.937 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:39 smithi123 ceph-mon[34645]: Deploying daemon osd.2 on smithi123 2024-09-21T02:53:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:39 smithi018 ceph-mon[34587]: Deploying daemon osd.2 on smithi123 2024-09-21T02:53:40.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:39 smithi016 ceph-mon[31253]: Deploying daemon osd.2 on smithi123 2024-09-21T02:53:41.072 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:40 smithi123 ceph-mon[34645]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:40 smithi018 ceph-mon[34587]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:41.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:40 smithi016 ceph-mon[31253]: pgmap v57: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:43.175 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:42 smithi123 ceph-mon[34645]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:42 smithi018 ceph-mon[34587]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:42 smithi016 ceph-mon[31253]: pgmap v58: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:45.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:44 smithi123 ceph-mon[34645]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:45.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:45.079 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:45.080 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:44 smithi018 ceph-mon[34587]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:44 smithi016 ceph-mon[31253]: pgmap v59: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:45.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:46 smithi018 ceph-mon[34587]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:46 smithi123 ceph-mon[34645]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:47.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:46 smithi016 ceph-mon[31253]: pgmap v60: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:48.400 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:48.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:48.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:48.401 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:48.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: pgmap v61: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:48.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.683+0000 7f50f8ff9640 1 -- 172.21.15.123:0/2985857740 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f50fc10ab30 con 0x7f50e40b64d0 2024-09-21T02:53:48.685 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 2 on host 'smithi123' 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 msgr2=0x7f50e40b89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 0x7f50e40b89e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f50ec005fd0 tx=0x7f50ec005850 comp rx=0 tx=0).stop 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 msgr2=0x7f50fc19fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f50f4005cc0 tx=0x7f50f402f380 comp rx=0 tx=0).stop 2024-09-21T02:53:48.687 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 shutdown_connections 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50e40b64d0 0x7f50e40b89e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50fc1083c0 0x7f50fc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50fc108d10 0x7f50fc19fc50 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 --2- 172.21.15.123:0/2985857740 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50fc102a80 0x7f50fc19a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 >> 172.21.15.123:0/2985857740 conn(0x7f50fc079740 msgr2=0x7f50fc104390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 shutdown_connections 2024-09-21T02:53:48.688 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:53:48.686+0000 7f5101f1d640 1 -- 172.21.15.123:0/2985857740 wait complete. 2024-09-21T02:53:49.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:49 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:49.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:49 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:49.342 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:49 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:49.344 DEBUG:teuthology.orchestra.run.smithi123:osd.2> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.2.service 2024-09-21T02:53:49.346 INFO:tasks.cephadm:Deploying osd.3 on smithi016 with /dev/vg_nvme/lv_3... 2024-09-21T02:53:49.346 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:53:49.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:49 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:49.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:49 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:49.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:49 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:49.614 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:49.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:49 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:53:49.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:49 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:49.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:49 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:50 smithi123 ceph-mon[34645]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:50 smithi123 ceph-mon[34645]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:50 smithi123 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:50.971 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T02:53:50.651+0000 7f0f6ac69640 -1 osd.2 0 log_to_monitors true 2024-09-21T02:53:50.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:50 smithi016 ceph-mon[31253]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:50.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:50 smithi016 ceph-mon[31253]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:50.982 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:50 smithi016 ceph-mon[31253]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:50 smithi018 ceph-mon[34587]: pgmap v62: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:50 smithi018 ceph-mon[34587]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:50 smithi018 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]} : dispatch 2024-09-21T02:53:51.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:51 smithi016 ceph-mon[31253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:53:51.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:51 smithi016 ceph-mon[31253]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:53:51.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:51 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:51.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:51 smithi016 ceph-mon[31253]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:51.921 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:51 smithi016 ceph-mon[31253]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:51.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:51 smithi123 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:53:51.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:51 smithi123 ceph-mon[34645]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:53:51.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:51 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:51.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:51 smithi123 ceph-mon[34645]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:51.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:51 smithi123 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:51 smithi018 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-09-21T02:53:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:51 smithi018 ceph-mon[34587]: osdmap e17: 3 total, 2 up, 3 in 2024-09-21T02:53:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:51 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:51 smithi018 ceph-mon[34587]: from='osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537]' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:51 smithi018 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd={"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:53:52.224 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:53:52.250 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_3 2024-09-21T02:53:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.908 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.909 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:52 smithi016 ceph-mon[31253]: from='osd.2 ' entity='osd.2' 2024-09-21T02:53:52.970 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T02:53:52.602+0000 7f0f66beb640 -1 osd.2 0 waiting for initial osdmap 2024-09-21T02:53:52.970 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T02:53:52.717+0000 7f0f5e9e1640 -1 osd.2 18 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:52.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:52 smithi123 ceph-mon[34645]: from='osd.2 ' entity='osd.2' 2024-09-21T02:53:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 -- 172.21.15.16:0/1475947387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a0102a80 msgr2=0x7f91a0102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 --2- 172.21.15.16:0/1475947387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a0102a80 0x7f91a0102e90 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f919c009900 tx=0x7f919c02a410 comp rx=0 tx=0).stop 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 -- 172.21.15.16:0/1475947387 shutdown_connections 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 --2- 172.21.15.16:0/1475947387 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 0x7f91a010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 --2- 172.21.15.16:0/1475947387 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a01083c0 0x7f91a01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 --2- 172.21.15.16:0/1475947387 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a0102a80 0x7f91a0102e90 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 -- 172.21.15.16:0/1475947387 >> 172.21.15.16:0/1475947387 conn(0x7f91a0079740 msgr2=0x7f91a0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 -- 172.21.15.16:0/1475947387 shutdown_connections 2024-09-21T02:53:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.003+0000 7f91a7380640 1 -- 172.21.15.16:0/1475947387 wait complete. 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 Processor -- start 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 -- start start 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a0102a80 0x7f91a019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 0x7f91a019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0112980 con 0x7f91a01083c0 2024-09-21T02:53:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0112800 con 0x7f91a0102a80 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a7380640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f91a0112b00 con 0x7f91a0108d10 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:55076/0 (socket says 172.21.15.16:55076) 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 -- 172.21.15.16:0/684140805 learned_addr learned my addr 172.21.15.16:0/684140805 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a511c640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a0102a80 0x7f91a019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a591d640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 0x7f91a019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 -- 172.21.15.16:0/684140805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 msgr2=0x7f91a019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 0x7f91a019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 -- 172.21.15.16:0/684140805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a0102a80 msgr2=0x7f91a019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a0102a80 0x7f91a019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.004+0000 7f91a491b640 1 -- 172.21.15.16:0/684140805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f91a01a0450 con 0x7f91a01083c0 2024-09-21T02:53:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91a491b640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f919000d770 tx=0x7f919000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190017070 con 0x7f91a01083c0 2024-09-21T02:53:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9190009c30 con 0x7f91a01083c0 2024-09-21T02:53:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919000f620 con 0x7f91a01083c0 2024-09-21T02:53:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f91a01a0740 con 0x7f91a01083c0 2024-09-21T02:53:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.005+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91a01a6ea0 con 0x7f91a01083c0 2024-09-21T02:53:53.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.006+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f91a0103d00 con 0x7f91a01083c0 2024-09-21T02:53:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.007+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 19) ==== 154473+0+0 (secure 0 0 0) 0x7f919001a070 con 0x7f91a01083c0 2024-09-21T02:53:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.008+0000 7f91967fc640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 0x7f918c0b8a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:53:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.008+0000 7f91a511c640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 0x7f918c0b8a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:53:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.008+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(18..18 src has 1..18) ==== 2786+0+0 (secure 0 0 0) 0x7f91900e5720 con 0x7f91a01083c0 2024-09-21T02:53:53.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.009+0000 7f91a511c640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 0x7f918c0b8a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f919c031440 tx=0x7f919c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:53:53.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.011+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9190009de0 con 0x7f91a01083c0 2024-09-21T02:53:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: pgmap v64: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:53:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: osdmap e18: 3 total, 2 up, 3 in 2024-09-21T02:53:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:53.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:52 smithi018 ceph-mon[34587]: from='osd.2 ' entity='osd.2' 2024-09-21T02:53:53.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:53.211+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f91a0106360 con 0x7f918c0b6550 2024-09-21T02:53:53.919 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:53:53.919 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:53:53.919 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi123 2024-09-21T02:53:53.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T02:53:53.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:53.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:53.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:53.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi123 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:53.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi123 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi123 to 17088M 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:53:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:53:54.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] boot 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:53:54.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:55.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:55.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:55.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:55.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] boot 2024-09-21T02:53:55.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:53:55.010 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: pgmap v66: 0 pgs: ; 0 B data, 53 MiB used, 179 GiB / 179 GiB avail 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: from='client.14520 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: osd.2 [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] boot 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: osdmap e19: 3 total, 3 up, 3 in 2024-09-21T02:53:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 2} : dispatch 2024-09-21T02:53:55.974 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:55 smithi123 ceph-mon[34645]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:53:55.975 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:55.975 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43676]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:55.975 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43676]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:55.975 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43676]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:55.975 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43676]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:55.990 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:55 smithi016 ceph-mon[31253]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:53:55.990 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:55.990 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50161]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:55.990 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50161]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:55.990 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50161]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:55.991 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50161]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:55.991 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50165]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:55.991 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50165]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:55.991 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50165]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:55.991 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 02:53:55 smithi016 sudo[50165]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.093 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:55 smithi018 ceph-mon[34587]: osdmap e20: 3 total, 3 up, 3 in 2024-09-21T02:53:56.093 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42811]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42811]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42811]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42811]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42814]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42814]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42814]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.094 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 02:53:55 smithi018 sudo[42814]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50174]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50174]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50174]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50174]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50177]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50177]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50177]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 sudo[50177]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42817]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42817]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42817]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42817]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42820]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42820]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42820]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 sudo[42820]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.470 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43679]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:56.470 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43679]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.471 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43679]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.471 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 02:53:55 smithi123 sudo[43679]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43682]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43682]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43682]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43682]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43685]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43685]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43685]: pam_unix(sudo:session): session opened for user root(uid=0) by (uid=167) 2024-09-21T02:53:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 sudo[43685]: pam_unix(sudo:session): session closed for user root 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: pgmap v69: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"} : dispatch 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"}]': finished 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:56.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:56.972 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:56.973 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3740186176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: pgmap v69: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"} : dispatch 2024-09-21T02:53:57.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"}]': finished 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3740186176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: pgmap v69: 0 pgs: ; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: osdmap e21: 3 total, 3 up, 3 in 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true} : dispatch 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"} : dispatch 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2062415210' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8bdffca0-2cd0-41f5-be85-ebe805e4a696"}]': finished 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: osdmap e22: 4 total, 3 up, 4 in 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-09-21T02:53:57.185 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "a"} : dispatch 2024-09-21T02:53:57.185 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "b"} : dispatch 2024-09-21T02:53:57.185 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mon metadata", "id": "c"} : dispatch 2024-09-21T02:53:57.185 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-09-21T02:53:57.185 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3740186176' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:53:57.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:53:57.761+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f91900a9210 con 0x7f91a01083c0 2024-09-21T02:53:58.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:57 smithi016 ceph-mon[31253]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:53:58.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:58.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:57 smithi018 ceph-mon[34587]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:53:58.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:58.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:57 smithi123 ceph-mon[34645]: osdmap e23: 4 total, 3 up, 4 in 2024-09-21T02:53:58.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:53:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:58 smithi016 ceph-mon[31253]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:53:58 smithi016 ceph-mon[31253]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T02:53:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:58 smithi018 ceph-mon[34587]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:53:58 smithi018 ceph-mon[34587]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T02:53:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:58 smithi123 ceph-mon[34645]: pgmap v73: 1 pgs: 1 unknown; 0 B data, 88 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:53:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:53:58 smithi123 ceph-mon[34645]: mgrmap e20: x(active, since 2m), standbys: z, y 2024-09-21T02:54:00.983 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:00 smithi016 ceph-mon[31253]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:00 smithi018 ceph-mon[34587]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:00 smithi123 ceph-mon[34645]: pgmap v74: 1 pgs: 1 active+clean; 577 KiB data, 89 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:02.946 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:02 smithi016 ceph-mon[31253]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:02 smithi018 ceph-mon[34587]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:03.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:02 smithi123 ceph-mon[34645]: pgmap v75: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:04.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:04 smithi016 ceph-mon[31253]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:04.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:54:04.898 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:05.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:04 smithi123 ceph-mon[34645]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:05.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:54:05.023 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:04 smithi018 ceph-mon[34587]: pgmap v76: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.3"} : dispatch 2024-09-21T02:54:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:05.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:05 smithi016 ceph-mon[31253]: Deploying daemon osd.3 on smithi016 2024-09-21T02:54:06.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:05 smithi018 ceph-mon[34587]: Deploying daemon osd.3 on smithi016 2024-09-21T02:54:06.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:05 smithi123 ceph-mon[34645]: Deploying daemon osd.3 on smithi016 2024-09-21T02:54:07.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:06 smithi016 ceph-mon[31253]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:06 smithi018 ceph-mon[34587]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:06 smithi123 ceph-mon[34645]: pgmap v77: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:09.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:08 smithi018 ceph-mon[34587]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:09.184 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:08 smithi016 ceph-mon[31253]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:09.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:08 smithi123 ceph-mon[34645]: pgmap v78: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:11.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:10 smithi016 ceph-mon[31253]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:10 smithi018 ceph-mon[34587]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:10 smithi123 ceph-mon[34645]: pgmap v79: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:12.243 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:12.243 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:12.243 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:13.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:12 smithi016 ceph-mon[31253]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:13.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:12 smithi018 ceph-mon[34587]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:13.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:12 smithi123 ceph-mon[34645]: pgmap v80: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.869+0000 7f91967fc640 1 -- 172.21.15.16:0/684140805 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f91a0106360 con 0x7f918c0b6550 2024-09-21T02:54:14.871 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 3 on host 'smithi016' 2024-09-21T02:54:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 msgr2=0x7f918c0b8a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 0x7f918c0b8a60 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f919c031440 tx=0x7f919c009560 comp rx=0 tx=0).stop 2024-09-21T02:54:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 msgr2=0x7f91a019aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f919000d770 tx=0x7f919000dc30 comp rx=0 tx=0).stop 2024-09-21T02:54:14.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 shutdown_connections 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f918c0b6550 0x7f918c0b8a60 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f91a01083c0 0x7f91a019aa80 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91a0108d10 0x7f91a019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 --2- 172.21.15.16:0/684140805 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f91a0102a80 0x7f91a019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 >> 172.21.15.16:0/684140805 conn(0x7f91a0079740 msgr2=0x7f91a0101b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 shutdown_connections 2024-09-21T02:54:14.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:54:14.872+0000 7f91a7380640 1 -- 172.21.15.16:0/684140805 wait complete. 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: pgmap v81: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:14.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:14.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:14.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.492 DEBUG:teuthology.orchestra.run.smithi016:osd.3> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.3.service 2024-09-21T02:54:15.494 INFO:tasks.cephadm:Deploying osd.4 on smithi018 with /dev/vg_nvme/lv_3... 2024-09-21T02:54:15.494 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:54:15.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:15.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.732 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:15.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.749 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.759 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:54:15.879 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:15.879 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:15.879 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:16.765 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 02:54:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T02:54:16.661+0000 7f55db393640 -1 osd.3 0 log_to_monitors true 2024-09-21T02:54:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:16 smithi018 ceph-mon[34587]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:16 smithi018 ceph-mon[34587]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:54:17.191 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:16 smithi016 ceph-mon[31253]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:17.191 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:16 smithi016 ceph-mon[31253]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:54:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:16 smithi123 ceph-mon[34645]: pgmap v82: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:16 smithi123 ceph-mon[34645]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]} : dispatch 2024-09-21T02:54:18.016 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T02:54:17.878+0000 7f55d7315640 -1 osd.3 0 waiting for initial osdmap 2024-09-21T02:54:18.016 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T02:54:17.888+0000 7f55d3113640 -1 osd.3 25 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:54:18.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:54:18.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:54:18.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:54:18.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:18.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:18.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:17 smithi016 ceph-mon[31253]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:54:18.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:54:18.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:54:18.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:54:18.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:18.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:18.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:17 smithi018 ceph-mon[34587]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:54:18.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-09-21T02:54:18.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: osdmap e24: 4 total, 3 up, 4 in 2024-09-21T02:54:18.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd={"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:54:18.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:18.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: pgmap v84: 1 pgs: 1 active+clean; 577 KiB data, 90 MiB used, 268 GiB / 268 GiB avail 2024-09-21T02:54:18.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:17 smithi123 ceph-mon[34645]: from='osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:54:18.501 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:54:18.526 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi018:vg_nvme/lv_3 2024-09-21T02:54:18.780 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi016 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.168 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] boot 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:54:19.169 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi016 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] boot 2024-09-21T02:54:19.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:54:19.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 -- 172.21.15.18:0/2091336998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 msgr2=0x7fd56810d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:19.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/2091336998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56810d8d0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd558009900 tx=0x7fd55802a410 comp rx=0 tx=0).stop 2024-09-21T02:54:19.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 -- 172.21.15.18:0/2091336998 shutdown_connections 2024-09-21T02:54:19.263 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/2091336998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd56810de10 0x7fd5681102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:19.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/2091336998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56810d8d0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:19.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/2091336998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd568104860 0x7fd568104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:19.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 -- 172.21.15.18:0/2091336998 >> 172.21.15.18:0/2091336998 conn(0x7fd568079730 msgr2=0x7fd568079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:19.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 -- 172.21.15.18:0/2091336998 shutdown_connections 2024-09-21T02:54:19.264 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.262+0000 7fd56f5b2640 1 -- 172.21.15.18:0/2091336998 wait complete. 2024-09-21T02:54:19.265 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 Processor -- start 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 -- start start 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568104860 0x7fd56819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd56810de10 0x7fd56819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd568113ab0 con 0x7fd568104860 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd568113930 con 0x7fd568105290 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56f5b2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd568113c30 con 0x7fd56810de10 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56cb4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56cb4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:39066/0 (socket says 172.21.15.18:39066) 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56cb4d640 1 -- 172.21.15.18:0/3609861851 learned_addr learned my addr 172.21.15.18:0/3609861851 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.263+0000 7fd56d34e640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568104860 0x7fd56819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56db4f640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd56810de10 0x7fd56819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:19.266 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 -- 172.21.15.18:0/3609861851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd56810de10 msgr2=0x7fd56819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd56810de10 0x7fd56819fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 -- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568104860 msgr2=0x7fd56819a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568104860 0x7fd56819a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 -- 172.21.15.18:0/3609861851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd5681a0410 con 0x7fd568105290 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56cb4d640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd558005ab0 tx=0x7fd558003cd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55803f070 con 0x7fd568105290 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd558031d60 con 0x7fd568105290 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd558035030 con 0x7fd568105290 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd5681a06a0 con 0x7fd568105290 2024-09-21T02:54:19.267 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.264+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd568102650 con 0x7fd568105290 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: osdmap e25: 4 total, 3 up, 4 in 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi016 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:19.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi016 to 8546M 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: osd.3 [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] boot 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: osdmap e26: 4 total, 4 up, 4 in 2024-09-21T02:54:19.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 3} : dispatch 2024-09-21T02:54:19.269 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.265+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56806b4e0 con 0x7fd568105290 2024-09-21T02:54:19.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.267+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd558042070 con 0x7fd568105290 2024-09-21T02:54:19.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.268+0000 7fd55e7fc640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 0x7fd5540b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:19.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.268+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3917+0+0 (secure 0 0 0) 0x7fd558106d80 con 0x7fd568105290 2024-09-21T02:54:19.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.268+0000 7fd56d34e640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 0x7fd5540b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:19.270 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.269+0000 7fd56d34e640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 0x7fd5540b8af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd564005fd0 tx=0x7fd56400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:19.272 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.271+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd5580ce6a0 con 0x7fd568105290 2024-09-21T02:54:19.472 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:19.471+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fd568063800 con 0x7fd5540b65e0 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:20.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:20.172 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:19 smithi018 ceph-mon[34587]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:54:20.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:54:20.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:54:20.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:20.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:19 smithi123 ceph-mon[34645]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:54:20.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: pgmap v87: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: from='client.24386 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:20.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:19 smithi016 ceph-mon[31253]: osdmap e27: 4 total, 4 up, 4 in 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/178477125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"}]': finished 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:54:22.171 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:21 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:54:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:22.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/178477125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"}]': finished 2024-09-21T02:54:22.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:54:22.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:21 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: osdmap e28: 4 total, 4 up, 4 in 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: pgmap v90: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/178477125' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"} : dispatch 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "3ee9ba63-3aca-4171-b196-a3b9e633a6b0"}]': finished 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: osdmap e29: 5 total, 4 up, 5 in 2024-09-21T02:54:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:21 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:23.167 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/1593734571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/1593734571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/1593734571' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:23 smithi018 ceph-mon[34587]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:24.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:23 smithi123 ceph-mon[34645]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:24.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:23 smithi016 ceph-mon[31253]: pgmap v92: 1 pgs: 1 peering; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:26.327 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:26 smithi018 ceph-mon[34587]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:54:26.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:26 smithi123 ceph-mon[34645]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:54:26.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:26 smithi016 ceph-mon[31253]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:54:28.327 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:28 smithi018 ceph-mon[34587]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-21T02:54:28.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:28 smithi123 ceph-mon[34645]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-21T02:54:28.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:28 smithi016 ceph-mon[31253]: pgmap v94: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 80 KiB/s, 0 objects/s recovering 2024-09-21T02:54:30.417 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:30 smithi018 ceph-mon[34587]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T02:54:30.417 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:54:30.417 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:30.418 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:30 smithi018 ceph-mon[34587]: Deploying daemon osd.4 on smithi018 2024-09-21T02:54:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:30 smithi123 ceph-mon[34645]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T02:54:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:54:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:30 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:30 smithi123 ceph-mon[34645]: Deploying daemon osd.4 on smithi018 2024-09-21T02:54:30.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:30 smithi016 ceph-mon[31253]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 70 KiB/s, 0 objects/s recovering 2024-09-21T02:54:30.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.4"} : dispatch 2024-09-21T02:54:30.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:30 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:30.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:30 smithi016 ceph-mon[31253]: Deploying daemon osd.4 on smithi018 2024-09-21T02:54:32.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:32 smithi018 ceph-mon[34587]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:54:32.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:32 smithi123 ceph-mon[34645]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:54:32.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:32 smithi016 ceph-mon[31253]: pgmap v96: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:54:34.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:34 smithi016 ceph-mon[31253]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:54:34.349 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:34 smithi018 ceph-mon[34587]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:54:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:34 smithi123 ceph-mon[34645]: pgmap v97: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:54:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:36 smithi018 ceph-mon[34587]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:54:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:36 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:36 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:36 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:36.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:36 smithi123 ceph-mon[34645]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:54:36.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:36 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:36.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:36 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:36.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:36 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:36.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:36 smithi016 ceph-mon[31253]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:54:36.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:36 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:36.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:36 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:36.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:36 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:38.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:38.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:38.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: pgmap v99: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.010 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 4 on host 'smithi018' 2024-09-21T02:54:39.010 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.008+0000 7fd55e7fc640 1 -- 172.21.15.18:0/3609861851 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fd568063800 con 0x7fd5540b65e0 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 msgr2=0x7fd5540b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 0x7fd5540b8af0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd564005fd0 tx=0x7fd56400b390 comp rx=0 tx=0).stop 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 msgr2=0x7fd56819aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd558005ab0 tx=0x7fd558003cd0 comp rx=0 tx=0).stop 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 shutdown_connections 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd5540b65e0 0x7fd5540b8af0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd56810de10 0x7fd56819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:39.013 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd568105290 0x7fd56819aa40 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:39.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 --2- 172.21.15.18:0/3609861851 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd568104860 0x7fd56819a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:39.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 >> 172.21.15.18:0/3609861851 conn(0x7fd568079730 msgr2=0x7fd568100820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:39.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 shutdown_connections 2024-09-21T02:54:39.014 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:54:39.011+0000 7fd56f5b2640 1 -- 172.21.15.18:0/3609861851 wait complete. 2024-09-21T02:54:39.363 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:39.364 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.364 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:54:39.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:39.682 DEBUG:teuthology.orchestra.run.smithi018:osd.4> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.4.service 2024-09-21T02:54:39.684 INFO:tasks.cephadm:Deploying osd.5 on smithi123 with /dev/vg_nvme/lv_3... 2024-09-21T02:54:39.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_3 2024-09-21T02:54:39.939 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:54:40.374 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:40 smithi123 ceph-mon[34645]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:40.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:40 smithi018 ceph-mon[34587]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:40.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:40 smithi016 ceph-mon[31253]: pgmap v100: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:42.170 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 02:54:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T02:54:41.849+0000 7ff94107c640 -1 osd.4 0 log_to_monitors true 2024-09-21T02:54:42.371 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:54:42.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_3 2024-09-21T02:54:42.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:42 smithi123 ceph-mon[34645]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:42.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:42 smithi123 ceph-mon[34645]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.428 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:42 smithi123 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:42 smithi018 ceph-mon[34587]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:42.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:42 smithi018 ceph-mon[34587]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:42 smithi018 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:42 smithi016 ceph-mon[31253]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:42.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:42 smithi016 ceph-mon[31253]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:42 smithi016 ceph-mon[31253]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]} : dispatch 2024-09-21T02:54:42.653 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:54:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 -- 172.21.15.123:0/2815356038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 msgr2=0x7fc384110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/2815356038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc384110300 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fc37c009ca0 tx=0x7fc37c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.110 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 -- 172.21.15.123:0/2815356038 shutdown_connections 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/2815356038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc384110300 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/2815356038 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 0x7fc38410d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/2815356038 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 0x7fc384104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 -- 172.21.15.123:0/2815356038 >> 172.21.15.123:0/2815356038 conn(0x7fc384079730 msgr2=0x7fc384079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 -- 172.21.15.123:0/2815356038 shutdown_connections 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.106+0000 7fc38bd2d640 1 -- 172.21.15.123:0/2815356038 wait complete. 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 Processor -- start 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 -- start start 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 0x7fc38419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 0x7fc38419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc384113b30 con 0x7fc384104840 2024-09-21T02:54:43.111 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc3841139b0 con 0x7fc384105270 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38bd2d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc384113cb0 con 0x7fc38410ddf0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36864/0 (socket says 172.21.15.123:36864) 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 -- 172.21.15.123:0/1975483625 learned_addr learned my addr 172.21.15.123:0/1975483625 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc389ac9640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 0x7fc38419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc3892c8640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 0x7fc38419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 -- 172.21.15.123:0/1975483625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 msgr2=0x7fc38419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 0x7fc38419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 -- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 msgr2=0x7fc38419a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 0x7fc38419a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.107+0000 7fc38a2ca640 1 -- 172.21.15.123:0/1975483625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3841a0450 con 0x7fc38410ddf0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc38a2ca640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37c005cc0 tx=0x7fc37c02f380 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c003bf0 con 0x7fc38410ddf0 2024-09-21T02:54:43.112 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc37c003d90 con 0x7fc38410ddf0 2024-09-21T02:54:43.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3841a06e0 con 0x7fc38410ddf0 2024-09-21T02:54:43.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc37c02fb20 con 0x7fc38410ddf0 2024-09-21T02:54:43.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.108+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3841a6ea0 con 0x7fc38410ddf0 2024-09-21T02:54:43.113 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.109+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc384104c50 con 0x7fc38410ddf0 2024-09-21T02:54:43.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.110+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc37c004580 con 0x7fc38410ddf0 2024-09-21T02:54:43.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.111+0000 7fc37affd640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 0x7fc3700b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:54:43.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.111+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) ==== 4020+0+0 (secure 0 0 0) 0x7fc37c1077f0 con 0x7fc38410ddf0 2024-09-21T02:54:43.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.111+0000 7fc389ac9640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 0x7fc3700b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:54:43.114 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.112+0000 7fc389ac9640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 0x7fc3700b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc38406bc00 tx=0x7fc380009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:54:43.118 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.116+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc37c0cf140 con 0x7fc38410ddf0 2024-09-21T02:54:43.313 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:54:43.310+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fc384100c70 con 0x7fc3700b6560 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi018 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.313 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi018 to 8546M 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.314 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:43 smithi123 ceph-mon[34645]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi018 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi018 to 8546M 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-mon[34587]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:54:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-09-21T02:54:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: osdmap e30: 5 total, 4 up, 5 in 2024-09-21T02:54:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101]' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='osd.4 ' entity='osd.4' cmd={"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi018 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi018 to 8546M 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:54:43.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:43 smithi016 ceph-mon[31253]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:54:44.177 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T02:54:43.882+0000 7ff93cffe640 -1 osd.4 0 waiting for initial osdmap 2024-09-21T02:54:44.177 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 02:54:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T02:54:43.890+0000 7ff934df4640 -1 osd.4 31 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:44.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:44.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:44.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:44 smithi018 ceph-mon[34587]: from='osd.4 ' entity='osd.4' 2024-09-21T02:54:44.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:44.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:54:44.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:44.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:44 smithi123 ceph-mon[34645]: from='osd.4 ' entity='osd.4' 2024-09-21T02:54:44.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: pgmap v103: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:44.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: osdmap e31: 5 total, 4 up, 5 in 2024-09-21T02:54:44.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='client.24403 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:44.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:44 smithi016 ceph-mon[31253]: from='osd.4 ' entity='osd.4' 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] boot 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:54:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] boot 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:54:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:54:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:54:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: osd.4 [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] boot 2024-09-21T02:54:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: osdmap e32: 5 total, 5 up, 5 in 2024-09-21T02:54:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 4} : dispatch 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/1411614987' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"}]': finished 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T02:54:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:46.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:46.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/1411614987' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"}]': finished 2024-09-21T02:54:46.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T02:54:46.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:46.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: pgmap v106: 1 pgs: 1 active+clean; 577 KiB data, 116 MiB used, 357 GiB / 358 GiB avail 2024-09-21T02:54:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/1411614987' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"} : dispatch 2024-09-21T02:54:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "ff1034fc-1839-47d8-80c6-93e0271bd1ec"}]': finished 2024-09-21T02:54:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: osdmap e33: 6 total, 5 up, 6 in 2024-09-21T02:54:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:47 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/1798285833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:47 smithi018 ceph-mon[34587]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:54:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:47.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:47 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/1798285833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:47.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:47 smithi123 ceph-mon[34645]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:54:47.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:47.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:47 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/1798285833' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:54:47.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:47 smithi016 ceph-mon[31253]: osdmap e34: 6 total, 5 up, 6 in 2024-09-21T02:54:47.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:54:48.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:48 smithi123 ceph-mon[34645]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:48.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:48 smithi016 ceph-mon[31253]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:48.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:48 smithi018 ceph-mon[34587]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:50.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:50 smithi123 ceph-mon[34645]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:50.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:50 smithi016 ceph-mon[31253]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:50.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:50 smithi018 ceph-mon[34587]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:52.466 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:52 smithi123 ceph-mon[34645]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:52.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:52 smithi016 ceph-mon[31253]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:52 smithi018 ceph-mon[34587]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:54 smithi123 ceph-mon[34645]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:54:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:54.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:54 smithi016 ceph-mon[31253]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:54.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:54:54.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:54 smithi018 ceph-mon[34587]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.5"} : dispatch 2024-09-21T02:54:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:54:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:55 smithi123 ceph-mon[34645]: Deploying daemon osd.5 on smithi123 2024-09-21T02:54:55.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:55 smithi016 ceph-mon[31253]: Deploying daemon osd.5 on smithi123 2024-09-21T02:54:55.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:55 smithi018 ceph-mon[34587]: Deploying daemon osd.5 on smithi123 2024-09-21T02:54:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:56 smithi123 ceph-mon[34645]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:56.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:56 smithi016 ceph-mon[31253]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:56 smithi018 ceph-mon[34587]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:58.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:54:58 smithi123 ceph-mon[34645]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:58.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:54:58 smithi016 ceph-mon[31253]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:54:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:54:58 smithi018 ceph-mon[34587]: pgmap v114: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:00.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:00 smithi123 ceph-mon[34645]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:00.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:00 smithi016 ceph-mon[31253]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:00 smithi018 ceph-mon[34587]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:01.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:01.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:01.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:02 smithi123 ceph-mon[34645]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:02.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:02 smithi016 ceph-mon[31253]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:02 smithi018 ceph-mon[34587]: pgmap v116: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:03.982 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.980+0000 7fc37affd640 1 -- 172.21.15.123:0/1975483625 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc384100c70 con 0x7fc3700b6560 2024-09-21T02:55:03.982 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 5 on host 'smithi123' 2024-09-21T02:55:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 msgr2=0x7fc3700b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 0x7fc3700b8a70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc38406bc00 tx=0x7fc380009110 comp rx=0 tx=0).stop 2024-09-21T02:55:03.984 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 msgr2=0x7fc38419fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc37c005cc0 tx=0x7fc37c02f380 comp rx=0 tx=0).stop 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 shutdown_connections 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3700b6560 0x7fc3700b8a70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc38410ddf0 0x7fc38419fd20 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc384105270 0x7fc38419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 --2- 172.21.15.123:0/1975483625 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc384104840 0x7fc38419a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 >> 172.21.15.123:0/1975483625 conn(0x7fc384079730 msgr2=0x7fc3840ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 shutdown_connections 2024-09-21T02:55:03.985 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:03.983+0000 7fc38bd2d640 1 -- 172.21.15.123:0/1975483625 wait complete. 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:03 smithi018 ceph-mon[34587]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:03 smithi123 ceph-mon[34645]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:04.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:04.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:04.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:04.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:03 smithi016 ceph-mon[31253]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:04.575 DEBUG:teuthology.orchestra.run.smithi123:osd.5> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.5.service 2024-09-21T02:55:04.617 INFO:tasks.cephadm:Deploying osd.6 on smithi016 with /dev/vg_nvme/lv_2... 2024-09-21T02:55:04.618 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:55:04.887 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:05.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:05.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:05.255 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:06.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:05 smithi018 ceph-mon[34587]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:06.192 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:05 smithi016 ceph-mon[31253]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:06.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:05 smithi123 ceph-mon[34645]: pgmap v118: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:06.220 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 02:55:06 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T02:55:06.054+0000 7fde37b30640 -1 osd.5 0 log_to_monitors true 2024-09-21T02:55:07.056 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:06 smithi123 ceph-mon[34645]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.056 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:06 smithi123 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:06 smithi018 ceph-mon[34587]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:06 smithi018 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:06 smithi016 ceph-mon[31253]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:06 smithi016 ceph-mon[31253]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]} : dispatch 2024-09-21T02:55:07.732 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:55:07.762 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_2 2024-09-21T02:55:08.039 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:55:08.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi123 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:08.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:08.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi123 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T02:55:08.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:08.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:08.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-09-21T02:55:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: osdmap e35: 6 total, 5 up, 6 in 2024-09-21T02:55:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085]' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd={"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi123 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi123 to 8544M 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:08.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 -- 172.21.15.16:0/1201144162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c108cf0 msgr2=0x7f392c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 --2- 172.21.15.16:0/1201144162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c108cf0 0x7f392c10f520 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f3924009ca0 tx=0x7f392402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 -- 172.21.15.16:0/1201144162 shutdown_connections 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 --2- 172.21.15.16:0/1201144162 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c108cf0 0x7f392c10f520 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 --2- 172.21.15.16:0/1201144162 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 0x7f392c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 --2- 172.21.15.16:0/1201144162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c102a60 0x7f392c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 -- 172.21.15.16:0/1201144162 >> 172.21.15.16:0/1201144162 conn(0x7f392c079730 msgr2=0x7f392c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.474+0000 7f3930aa0640 1 -- 172.21.15.16:0/1201144162 shutdown_connections 2024-09-21T02:55:08.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.475+0000 7f3930aa0640 1 -- 172.21.15.16:0/1201144162 wait complete. 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.475+0000 7f3930aa0640 1 Processor -- start 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.475+0000 7f3930aa0640 1 -- start start 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 0x7f392c19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c108cf0 0x7f392c19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f392c112960 con 0x7f392c102a60 2024-09-21T02:55:08.479 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f392c1127e0 con 0x7f392c1083a0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3930aa0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f392c112ae0 con 0x7f392c108cf0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53302/0 (socket says 172.21.15.16:53302) 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 -- 172.21.15.16:0/3715673469 learned_addr learned my addr 172.21.15.16:0/3715673469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392ad9d640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c108cf0 0x7f392c19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f3929d9b640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 0x7f392c19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 -- 172.21.15.16:0/3715673469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c108cf0 msgr2=0x7f392c19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c108cf0 0x7f392c19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 -- 172.21.15.16:0/3715673469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 msgr2=0x7f392c19a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 0x7f392c19a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 -- 172.21.15.16:0/3715673469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f392c1a03b0 con 0x7f392c102a60 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.476+0000 7f392a59c640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f391c00e9f0 tx=0x7f391c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.477+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c00cd80 con 0x7f392c102a60 2024-09-21T02:55:08.480 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.477+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f392c1a06a0 con 0x7f392c102a60 2024-09-21T02:55:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.477+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f391c0054d0 con 0x7f392c102a60 2024-09-21T02:55:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.477+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f391c010690 con 0x7f392c102a60 2024-09-21T02:55:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.477+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f392c0ff550 con 0x7f392c102a60 2024-09-21T02:55:08.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.478+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f392c102e70 con 0x7f392c102a60 2024-09-21T02:55:08.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.480+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f391c004e50 con 0x7f392c102a60 2024-09-21T02:55:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.481+0000 7f391b7fe640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 0x7f39140b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.481+0000 7f3929d9b640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 0x7f39140b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.481+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(36..36 src has 1..36) ==== 4468+0+0 (secure 0 0 0) 0x7f391c0e6ae0 con 0x7f392c102a60 2024-09-21T02:55:08.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.482+0000 7f3929d9b640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 0x7f39140b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f3920005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:08.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.485+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f391c056710 con 0x7f392c102a60 2024-09-21T02:55:08.686 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:08.684+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f392c063800 con 0x7f39140b6560 2024-09-21T02:55:09.220 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T02:55:08.839+0000 7fde33ab2640 -1 osd.5 0 waiting for initial osdmap 2024-09-21T02:55:09.220 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T02:55:08.982+0000 7fde2b8a8640 -1 osd.5 36 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:08 smithi123 ceph-mon[34645]: from='osd.5 ' entity='osd.5' 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:09.222 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:08 smithi016 ceph-mon[31253]: from='osd.5 ' entity='osd.5' 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: osdmap e36: 6 total, 5 up, 6 in 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='client.14586 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:09.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:08 smithi018 ceph-mon[34587]: from='osd.5 ' entity='osd.5' 2024-09-21T02:55:10.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:55:10.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:55:10.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T02:55:10.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:09 smithi123 ceph-mon[34645]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:10.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:55:10.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:55:10.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T02:55:10.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:09 smithi016 ceph-mon[31253]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:55:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:55:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: osdmap e37: 6 total, 5 up, 6 in 2024-09-21T02:55:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:10.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:09 smithi018 ceph-mon[34587]: pgmap v123: 1 pgs: 1 peering; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-09-21T02:55:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:10 smithi123 ceph-mon[34645]: osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] boot 2024-09-21T02:55:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:10 smithi123 ceph-mon[34645]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:55:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:11.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:10 smithi123 ceph-mon[34645]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:55:11.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:10 smithi016 ceph-mon[31253]: osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] boot 2024-09-21T02:55:11.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:10 smithi016 ceph-mon[31253]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:55:11.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:11.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:10 smithi016 ceph-mon[31253]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:55:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:10 smithi018 ceph-mon[34587]: osd.5 [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] boot 2024-09-21T02:55:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:10 smithi018 ceph-mon[34587]: osdmap e38: 6 total, 6 up, 6 in 2024-09-21T02:55:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 5} : dispatch 2024-09-21T02:55:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:10 smithi018 ceph-mon[34587]: osdmap e39: 6 total, 6 up, 6 in 2024-09-21T02:55:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:11 smithi123 ceph-mon[34645]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:12.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:11 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"} : dispatch 2024-09-21T02:55:12.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:11 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"}]': finished 2024-09-21T02:55:12.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:11 smithi123 ceph-mon[34645]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:55:12.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:12.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:11 smithi016 ceph-mon[31253]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:12.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:11 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"} : dispatch 2024-09-21T02:55:12.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:11 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"}]': finished 2024-09-21T02:55:12.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:11 smithi016 ceph-mon[31253]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:55:12.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:11 smithi018 ceph-mon[34587]: pgmap v126: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:11 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"} : dispatch 2024-09-21T02:55:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:11 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1083404959' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "c0460abd-d09c-42d4-90e1-fb8ce20982f3"}]': finished 2024-09-21T02:55:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:11 smithi018 ceph-mon[34587]: osdmap e40: 7 total, 6 up, 7 in 2024-09-21T02:55:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:13.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:12 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/536303190' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:13.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:12 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/536303190' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:12 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/536303190' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:14.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:13 smithi123 ceph-mon[34645]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:14.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:13 smithi016 ceph-mon[31253]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:13 smithi018 ceph-mon[34587]: pgmap v128: 1 pgs: 1 peering; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:16.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:16 smithi016 ceph-mon[31253]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:55:16.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:16 smithi018 ceph-mon[34587]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:55:16.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:16 smithi123 ceph-mon[34645]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 182 MiB used, 536 GiB / 536 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:55:18.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:18 smithi018 ceph-mon[34587]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T02:55:18.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:18 smithi123 ceph-mon[34645]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T02:55:18.492 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:18 smithi016 ceph-mon[31253]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 81 KiB/s, 0 objects/s recovering 2024-09-21T02:55:20.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:20 smithi016 ceph-mon[31253]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T02:55:20.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:55:20.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:20.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:20 smithi018 ceph-mon[34587]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T02:55:20.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:55:20.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:20.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:20 smithi123 ceph-mon[34645]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 71 KiB/s, 0 objects/s recovering 2024-09-21T02:55:20.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.6"} : dispatch 2024-09-21T02:55:20.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:21.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:21 smithi018 ceph-mon[34587]: Deploying daemon osd.6 on smithi016 2024-09-21T02:55:21.437 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:21 smithi016 ceph-mon[31253]: Deploying daemon osd.6 on smithi016 2024-09-21T02:55:21.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:21 smithi123 ceph-mon[34645]: Deploying daemon osd.6 on smithi016 2024-09-21T02:55:22.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:22 smithi016 ceph-mon[31253]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:55:22.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:22 smithi018 ceph-mon[34587]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:55:22.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:22 smithi123 ceph-mon[34645]: pgmap v132: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:55:24.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:24 smithi018 ceph-mon[34587]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T02:55:24.462 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:24 smithi016 ceph-mon[31253]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T02:55:24.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:24 smithi123 ceph-mon[34645]: pgmap v133: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 49 KiB/s, 0 objects/s recovering 2024-09-21T02:55:26.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:26 smithi016 ceph-mon[31253]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:55:26.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:26 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:26.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:26 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:26.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:26 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:26 smithi018 ceph-mon[34587]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:55:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:26 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:26 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:26.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:26 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:26.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:26 smithi123 ceph-mon[34645]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:55:26.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:26 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:26.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:26 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:26.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:26 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:28.441 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:28 smithi018 ceph-mon[34587]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:28.470 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:28 smithi016 ceph-mon[31253]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:28.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:28 smithi123 ceph-mon[34645]: pgmap v135: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:30.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.154+0000 7f391b7fe640 1 -- 172.21.15.16:0/3715673469 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f392c063800 con 0x7f39140b6560 2024-09-21T02:55:30.156 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 6 on host 'smithi016' 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 msgr2=0x7f39140b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 0x7f39140b8a70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f3920005fd0 tx=0x7f3920005850 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 msgr2=0x7f392c19a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f391c00e9f0 tx=0x7f391c00edf0 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 shutdown_connections 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39140b6560 0x7f39140b8a70 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f392c108cf0 0x7f392c19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f392c1083a0 0x7f392c19a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 --2- 172.21.15.16:0/3715673469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f392c102a60 0x7f392c19a440 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 >> 172.21.15.16:0/3715673469 conn(0x7f392c079730 msgr2=0x7f392c06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 shutdown_connections 2024-09-21T02:55:30.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:55:30.157+0000 7f3930aa0640 1 -- 172.21.15.16:0/3715673469 wait complete. 2024-09-21T02:55:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:30.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:30.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:30.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:30 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: pgmap v136: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:30.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:30.856 DEBUG:teuthology.orchestra.run.smithi016:osd.6> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.6.service 2024-09-21T02:55:30.859 INFO:tasks.cephadm:Deploying osd.7 on smithi018 with /dev/vg_nvme/lv_2... 2024-09-21T02:55:30.859 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:55:31.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:31.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:31.075 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:31.123 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:55:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:32.238 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:32 smithi016 ceph-mon[31253]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:32.239 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:32 smithi016 ceph-mon[31253]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:55:32.239 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 02:55:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T02:55:31.927+0000 7f0da946d640 -1 osd.6 0 log_to_monitors true 2024-09-21T02:55:32.271 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:32 smithi018 ceph-mon[34587]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:32.271 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:32 smithi018 ceph-mon[34587]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:55:32.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:32 smithi123 ceph-mon[34645]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:32.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:32 smithi123 ceph-mon[34645]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]} : dispatch 2024-09-21T02:55:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:33 smithi018 ceph-mon[34587]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:55:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:33 smithi018 ceph-mon[34587]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:55:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:33 smithi018 ceph-mon[34587]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:55:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:33 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-mon[31253]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:55:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-mon[31253]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:55:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-mon[31253]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:55:33.413 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:33.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:33 smithi123 ceph-mon[34645]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-09-21T02:55:33.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:33 smithi123 ceph-mon[34645]: osdmap e41: 7 total, 6 up, 7 in 2024-09-21T02:55:33.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:33 smithi123 ceph-mon[34645]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd={"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:55:33.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:33 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:33.631 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:55:33.656 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi018:vg_nvme/lv_2 2024-09-21T02:55:33.911 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:55:34.015 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T02:55:33.848+0000 7f0da53ef640 -1 osd.6 0 waiting for initial osdmap 2024-09-21T02:55:34.015 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 02:55:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T02:55:33.858+0000 7f0d9d1e5640 -1 osd.6 42 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:34.265 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:34 smithi018 ceph-mon[34587]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' 2024-09-21T02:55:34.364 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- 172.21.15.18:0/639574252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 msgr2=0x7f38bc10d850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 --2- 172.21.15.18:0/639574252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc10d850 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f38ac009900 tx=0x7f38ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- 172.21.15.18:0/639574252 shutdown_connections 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 --2- 172.21.15.18:0/639574252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc10de10 0x7f38bc110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 --2- 172.21.15.18:0/639574252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc10d850 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 --2- 172.21.15.18:0/639574252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc1049b0 0x7f38bc104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- 172.21.15.18:0/639574252 >> 172.21.15.18:0/639574252 conn(0x7f38bc079720 msgr2=0x7f38bc079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- 172.21.15.18:0/639574252 shutdown_connections 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- 172.21.15.18:0/639574252 wait complete. 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 Processor -- start 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.360+0000 7f38c42b3640 1 -- start start 2024-09-21T02:55:34.365 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc1049b0 0x7f38bc19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc10de10 0x7f38bc19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc113a70 con 0x7f38bc10de10 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc1138f0 con 0x7f38bc105300 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38bc113bf0 con 0x7f38bc1049b0 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:33918/0 (socket says 172.21.15.18:33918) 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 -- 172.21.15.18:0/2087527557 learned_addr learned my addr 172.21.15.18:0/2087527557 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c204f640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc1049b0 0x7f38bc19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c2850640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc10de10 0x7f38bc19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:34.366 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 -- 172.21.15.18:0/2087527557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc1049b0 msgr2=0x7f38bc19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc1049b0 0x7f38bc19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 -- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc10de10 msgr2=0x7f38bc19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc10de10 0x7f38bc19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 -- 172.21.15.18:0/2087527557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38bc1a03b0 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c184e640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38ac006010 tx=0x7f38ac004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac03f030 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38bc1a0640 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38ac003aa0 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.361+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38ac035070 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.362+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38bc1a6e00 con 0x7f38bc105300 2024-09-21T02:55:34.367 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.363+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38bc06c2e0 con 0x7f38bc105300 2024-09-21T02:55:34.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.364+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f38ac042030 con 0x7f38bc105300 2024-09-21T02:55:34.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.365+0000 7f38b37fe640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 0x7f38a80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:34.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.365+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(42..42 src has 1..42) ==== 4900+0+0 (secure 0 0 0) 0x7f38ac106fd0 con 0x7f38bc105300 2024-09-21T02:55:34.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.365+0000 7f38c204f640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 0x7f38a80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:34.368 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.366+0000 7f38c204f640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 0x7f38a80b8a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06c9e0 tx=0x7f38b8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:34.369 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.368+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38ac0ce510 con 0x7f38bc105300 2024-09-21T02:55:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:55:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:55:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:34 smithi123 ceph-mon[34645]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: osdmap e42: 7 total, 6 up, 7 in 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: pgmap v140: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-09-21T02:55:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:34 smithi016 ceph-mon[31253]: from='osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295]' entity='osd.6' 2024-09-21T02:55:34.568 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:34.567+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f38bc108f40 con 0x7f38a80b6560 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi016 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] boot 2024-09-21T02:55:35.235 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:55:35.236 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi016 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] boot 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:55:35.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:35.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi016 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi016 to 5697M 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:35.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='client.24464 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: osd.6 [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] boot 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: osdmap e43: 7 total, 7 up, 7 in 2024-09-21T02:55:35.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 6} : dispatch 2024-09-21T02:55:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:36 smithi016 ceph-mon[31253]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:36 smithi016 ceph-mon[31253]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:55:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:36 smithi018 ceph-mon[34587]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:36 smithi018 ceph-mon[34587]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:55:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:36 smithi123 ceph-mon[34645]: pgmap v142: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:36 smithi123 ceph-mon[34645]: osdmap e44: 7 total, 7 up, 7 in 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/3668733470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"}]': finished 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:37 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/1820222521' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/3668733470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"}]': finished 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:38.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:37 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/1820222521' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:38.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/3668733470' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"} : dispatch 2024-09-21T02:55:38.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "1197b70e-2896-4524-8174-10c95b6b03c7"}]': finished 2024-09-21T02:55:38.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: osdmap e45: 8 total, 7 up, 8 in 2024-09-21T02:55:38.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:38.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:37 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/1820222521' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:55:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:38 smithi018 ceph-mon[34587]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:38 smithi123 ceph-mon[34645]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:39.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:38 smithi016 ceph-mon[31253]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:40 smithi018 ceph-mon[34587]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:40 smithi123 ceph-mon[34645]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:41.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:40 smithi016 ceph-mon[31253]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:43.173 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:42 smithi018 ceph-mon[34587]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:42 smithi123 ceph-mon[34645]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:42 smithi016 ceph-mon[31253]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:45.084 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:44 smithi018 ceph-mon[34587]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:44 smithi123 ceph-mon[34645]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:44 smithi016 ceph-mon[31253]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:46.148 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:55:46.148 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:46.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:55:46.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:46.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.7"} : dispatch 2024-09-21T02:55:46.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:46 smithi018 ceph-mon[34587]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:46 smithi018 ceph-mon[34587]: Deploying daemon osd.7 on smithi018 2024-09-21T02:55:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:46 smithi123 ceph-mon[34645]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:46 smithi123 ceph-mon[34645]: Deploying daemon osd.7 on smithi018 2024-09-21T02:55:47.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:46 smithi016 ceph-mon[31253]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:46 smithi016 ceph-mon[31253]: Deploying daemon osd.7 on smithi018 2024-09-21T02:55:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:48 smithi018 ceph-mon[34587]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:48 smithi123 ceph-mon[34645]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:49.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:48 smithi016 ceph-mon[31253]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:50 smithi018 ceph-mon[34587]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:50 smithi123 ceph-mon[34645]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:50 smithi016 ceph-mon[31253]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:52.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:51 smithi123 ceph-mon[34645]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:52.245 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:51 smithi018 ceph-mon[34587]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:52.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:51 smithi016 ceph-mon[31253]: pgmap v152: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:54.426 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.424+0000 7f38b37fe640 1 -- 172.21.15.18:0/2087527557 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f38bc108f40 con 0x7f38a80b6560 2024-09-21T02:55:54.426 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 7 on host 'smithi018' 2024-09-21T02:55:54.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 msgr2=0x7f38a80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:54.428 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 0x7f38a80b8a70 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f38bc06c9e0 tx=0x7f38b8005f90 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 msgr2=0x7f38bc19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f38ac006010 tx=0x7f38ac004060 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 shutdown_connections 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38a80b6560 0x7f38a80b8a70 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38bc10de10 0x7f38bc19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38bc105300 0x7f38bc19a9e0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 --2- 172.21.15.18:0/2087527557 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38bc1049b0 0x7f38bc19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 >> 172.21.15.18:0/2087527557 conn(0x7f38bc079720 msgr2=0x7f38bc1007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 shutdown_connections 2024-09-21T02:55:54.429 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:55:54.427+0000 7f38c42b3640 1 -- 172.21.15.18:0/2087527557 wait complete. 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:54.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:54.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: pgmap v153: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:54.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:54.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:54.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:54.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.189 DEBUG:teuthology.orchestra.run.smithi018:osd.7> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.7.service 2024-09-21T02:55:55.191 INFO:tasks.cephadm:Deploying osd.8 on smithi123 with /dev/vg_nvme/lv_2... 2024-09-21T02:55:55.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_2 2024-09-21T02:55:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.448 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:55:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:55:55.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:55.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:56.459 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:56 smithi123 ceph-mon[34645]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:56.567 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:56 smithi018 ceph-mon[34587]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:56.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:56 smithi016 ceph-mon[31253]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:57.182 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 02:55:56 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T02:55:56.921+0000 7f8297a20640 -1 osd.7 0 log_to_monitors true 2024-09-21T02:55:57.573 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:57 smithi018 ceph-mon[34587]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.574 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:57 smithi018 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.612 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:57 smithi123 ceph-mon[34645]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.612 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:57 smithi123 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:57 smithi016 ceph-mon[31253]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:57 smithi016 ceph-mon[31253]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]} : dispatch 2024-09-21T02:55:57.785 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:55:57.811 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_2 2024-09-21T02:55:58.063 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 -- 172.21.15.123:0/259993566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc004108cf0 msgr2=0x7fc00410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 --2- 172.21.15.123:0/259993566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc004108cf0 0x7fc00410f520 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbffc009900 tx=0x7fbffc02a410 comp rx=0 tx=0).stop 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 -- 172.21.15.123:0/259993566 shutdown_connections 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 --2- 172.21.15.123:0/259993566 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc004108cf0 0x7fc00410f520 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 --2- 172.21.15.123:0/259993566 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc0041083a0 0x7fc0041087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.552 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 --2- 172.21.15.123:0/259993566 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc004102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 -- 172.21.15.123:0/259993566 >> 172.21.15.123:0/259993566 conn(0x7fc004079730 msgr2=0x7fc004079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 -- 172.21.15.123:0/259993566 shutdown_connections 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.530+0000 7fc00c426640 1 -- 172.21.15.123:0/259993566 wait complete. 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 Processor -- start 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 -- start start 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc00419a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc004108cf0 0x7fc00419fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc004113a80 con 0x7fc004102a60 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc004113900 con 0x7fc004108cf0 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00c426640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc004113c00 con 0x7fc0041083a0 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc0099c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc0099c1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36096/0 (socket says 172.21.15.123:36096) 2024-09-21T02:55:58.553 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc0099c1640 1 -- 172.21.15.123:0/822908118 learned_addr learned my addr 172.21.15.123:0/822908118 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.531+0000 7fc00a9c3640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc004108cf0 0x7fc00419fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc00a1c2640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc00419a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 -- 172.21.15.123:0/822908118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc004108cf0 msgr2=0x7fc00419fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc004108cf0 0x7fc00419fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 -- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 msgr2=0x7fc00419a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc00419a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 -- 172.21.15.123:0/822908118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc0041a0320 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc00a1c2640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc00419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0099c1640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005fd0 tx=0x7fbff800dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8017070 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbff8009c30 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc0041a0610 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbff8003d30 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.532+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc0041a6d70 con 0x7fc0041083a0 2024-09-21T02:55:58.554 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.533+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc004102e70 con 0x7fc0041083a0 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.535+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbff8003ed0 con 0x7fc0041083a0 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.536+0000 7fc0037fe640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 0x7fbfec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.536+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 5332+0+0 (secure 0 0 0) 0x7fbff80e6bc0 con 0x7fc0041083a0 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.536+0000 7fc00a1c2640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 0x7fbfec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.537+0000 7fc00a1c2640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 0x7fbfec0b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff4009510 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:55:58.555 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.539+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbff8020030 con 0x7fc0041083a0 2024-09-21T02:55:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:55:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:58.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.729 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:55:58.728+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fc004063800 con 0x7fbfec0b6560 2024-09-21T02:55:58.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:58.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:58.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: pgmap v155: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: osdmap e46: 8 total, 7 up, 8 in 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231]' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd={"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:55:58.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:55:59.435 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T02:55:59.116+0000 7f82939a2640 -1 osd.7 0 waiting for initial osdmap 2024-09-21T02:55:59.435 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T02:55:59.129+0000 7f828b798640 -1 osd.7 47 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:55:59.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi018 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi018 to 5697M 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:59.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:55:59 smithi123 ceph-mon[34645]: from='osd.7 ' entity='osd.7' 2024-09-21T02:55:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi018 2024-09-21T02:55:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi018 to 5697M 2024-09-21T02:55:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:55:59.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:59.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:55:59 smithi016 ceph-mon[31253]: from='osd.7 ' entity='osd.7' 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi018 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi018 to 5697M 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: osdmap e47: 8 total, 7 up, 8 in 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='client.24469 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:55:59.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:55:59.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:55:59.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:55:59 smithi018 ceph-mon[34587]: from='osd.7 ' entity='osd.7' 2024-09-21T02:56:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:56:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:56:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:56:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] boot 2024-09-21T02:56:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:56:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:56:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:00 smithi123 ceph-mon[34645]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:56:00.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:56:00.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:56:00.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:56:00.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] boot 2024-09-21T02:56:00.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:56:00.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:56:00.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:00 smithi016 ceph-mon[31253]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:56:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:56:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:56:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 218 MiB used, 626 GiB / 626 GiB avail 2024-09-21T02:56:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: osd.7 [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] boot 2024-09-21T02:56:00.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: osdmap e48: 8 total, 8 up, 8 in 2024-09-21T02:56:00.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 7} : dispatch 2024-09-21T02:56:00.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:00 smithi018 ceph-mon[34587]: osdmap e49: 8 total, 8 up, 8 in 2024-09-21T02:56:01.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:01 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/1390943179' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:01 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:01 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"}]': finished 2024-09-21T02:56:01.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:01 smithi123 ceph-mon[34645]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:56:01.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:01 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:01.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:01 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/1390943179' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:01 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:01 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"}]': finished 2024-09-21T02:56:01.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:01 smithi016 ceph-mon[31253]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:56:01.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:01 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:01 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/1390943179' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:01 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"} : dispatch 2024-09-21T02:56:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:01 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df052aaa-1464-46d8-828f-99ce714b76c6"}]': finished 2024-09-21T02:56:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:01 smithi018 ceph-mon[34587]: osdmap e50: 9 total, 8 up, 9 in 2024-09-21T02:56:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:01 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:02.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:02 smithi123 ceph-mon[34645]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:02.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:02 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/3583927197' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:02.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:02 smithi123 ceph-mon[34645]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:56:02.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:02.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:02 smithi016 ceph-mon[31253]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/3583927197' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:02 smithi016 ceph-mon[31253]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:56:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:02 smithi018 ceph-mon[34587]: pgmap v162: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:02 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/3583927197' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:02 smithi018 ceph-mon[34587]: osdmap e51: 9 total, 8 up, 9 in 2024-09-21T02:56:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:04.722 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:04 smithi123 ceph-mon[34645]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:04.735 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:04 smithi018 ceph-mon[34587]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:04.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:04 smithi016 ceph-mon[31253]: pgmap v164: 1 pgs: 1 remapped; 577 KiB data, 648 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:06.692 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:06 smithi123 ceph-mon[34645]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:56:06.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:06 smithi016 ceph-mon[31253]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:56:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:06 smithi018 ceph-mon[34587]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 102 KiB/s, 0 objects/s recovering 2024-09-21T02:56:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:08 smithi123 ceph-mon[34645]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-09-21T02:56:08.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:08 smithi016 ceph-mon[31253]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-09-21T02:56:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:08 smithi018 ceph-mon[34587]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 173 KiB/s, 0 objects/s recovering 2024-09-21T02:56:09.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:56:09.697 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:09.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:56:09.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:09.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.8"} : dispatch 2024-09-21T02:56:09.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:10.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:10 smithi123 ceph-mon[34645]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T02:56:10.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:10 smithi123 ceph-mon[34645]: Deploying daemon osd.8 on smithi123 2024-09-21T02:56:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:10 smithi016 ceph-mon[31253]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T02:56:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:10 smithi016 ceph-mon[31253]: Deploying daemon osd.8 on smithi123 2024-09-21T02:56:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:10 smithi018 ceph-mon[34587]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 140 KiB/s, 0 objects/s recovering 2024-09-21T02:56:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:10 smithi018 ceph-mon[34587]: Deploying daemon osd.8 on smithi123 2024-09-21T02:56:12.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:12 smithi123 ceph-mon[34645]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T02:56:12.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:12 smithi016 ceph-mon[31253]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T02:56:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:12 smithi018 ceph-mon[34587]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 115 KiB/s, 0 objects/s recovering 2024-09-21T02:56:14.636 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:14 smithi123 ceph-mon[34645]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-21T02:56:14.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:14 smithi016 ceph-mon[31253]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-21T02:56:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:14 smithi018 ceph-mon[34587]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 103 KiB/s, 0 objects/s recovering 2024-09-21T02:56:15.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:15.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:15.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:15.800 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:15.801 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:15.801 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:15.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:15.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:15.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:16.684 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:16 smithi123 ceph-mon[34645]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:56:16.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:16 smithi016 ceph-mon[31253]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:56:16.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:16 smithi018 ceph-mon[34587]: pgmap v170: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:56:18.788 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.786+0000 7fc0037fe640 1 -- 172.21.15.123:0/822908118 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fc004063800 con 0x7fbfec0b6560 2024-09-21T02:56:18.788 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 8 on host 'smithi123' 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 msgr2=0x7fbfec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 0x7fbfec0b8a70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fbff4005fd0 tx=0x7fbff4009510 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 msgr2=0x7fc00419a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbff8005fd0 tx=0x7fbff800dde0 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 shutdown_connections 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbfec0b6560 0x7fbfec0b8a70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc004108cf0 0x7fc00419fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0041083a0 0x7fc00419a950 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 --2- 172.21.15.123:0/822908118 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc004102a60 0x7fc00419a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:18.790 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.788+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 >> 172.21.15.123:0/822908118 conn(0x7fc004079730 msgr2=0x7fc004104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.789+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 shutdown_connections 2024-09-21T02:56:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:56:18.789+0000 7fc00c426640 1 -- 172.21.15.123:0/822908118 wait complete. 2024-09-21T02:56:19.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:19.158 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: pgmap v171: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.510 DEBUG:teuthology.orchestra.run.smithi123:osd.8> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.8.service 2024-09-21T02:56:19.512 INFO:tasks.cephadm:Deploying osd.9 on smithi016 with /dev/vg_nvme/lv_1... 2024-09-21T02:56:19.512 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:56:19.784 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:19.904 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:19.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:19.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:19 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:20.020 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:20.020 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:20.020 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:19 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:20.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:20.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:20.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:19 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:21.001 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:20 smithi123 ceph-mon[34645]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:21.001 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:20 smithi016 ceph-mon[31253]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:20 smithi018 ceph-mon[34587]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:21.471 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 02:56:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T02:56:20.999+0000 7f55ab0a6640 -1 osd.8 0 log_to_monitors true 2024-09-21T02:56:21.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:21 smithi123 ceph-mon[34645]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:21.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:21 smithi123 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:22.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:21 smithi016 ceph-mon[31253]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:22.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:21 smithi016 ceph-mon[31253]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:21 smithi018 ceph-mon[34587]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:21 smithi018 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]} : dispatch 2024-09-21T02:56:22.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:56:22.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi016:vg_nvme/lv_1 2024-09-21T02:56:22.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:23.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:23.074 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 -- 172.21.15.16:0/3093234577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc102a70 msgr2=0x7f98bc102e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:23.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 --2- 172.21.15.16:0/3093234577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc102a70 0x7f98bc102e80 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f98ac009900 tx=0x7f98ac02a410 comp rx=0 tx=0).stop 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 -- 172.21.15.16:0/3093234577 shutdown_connections 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 --2- 172.21.15.16:0/3093234577 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 0x7f98bc10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 --2- 172.21.15.16:0/3093234577 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc1083b0 0x7f98bc1087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 --2- 172.21.15.16:0/3093234577 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc102a70 0x7f98bc102e80 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 -- 172.21.15.16:0/3093234577 >> 172.21.15.16:0/3093234577 conn(0x7f98bc079710 msgr2=0x7f98bc079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 -- 172.21.15.16:0/3093234577 shutdown_connections 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.167+0000 7f98c1abf640 1 -- 172.21.15.16:0/3093234577 wait complete. 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 Processor -- start 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 -- start start 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc102a70 0x7f98bc1993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 0x7f98bc19ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:23.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc1129c0 con 0x7f98bc1083b0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc112840 con 0x7f98bc108d00 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98c1abf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98bc112b40 con 0x7f98bc102a70 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98baffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33676/0 (socket says 172.21.15.16:33676) 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.168+0000 7f98baffd640 1 -- 172.21.15.16:0/1862391825 learned_addr learned my addr 172.21.15.16:0/1862391825 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98bb7fe640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc102a70 0x7f98bc1993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98bbfff640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 0x7f98bc19ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 -- 172.21.15.16:0/1862391825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc102a70 msgr2=0x7f98bc1993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc102a70 0x7f98bc1993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 -- 172.21.15.16:0/1862391825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 msgr2=0x7f98bc19ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 0x7f98bc19ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 -- 172.21.15.16:0/1862391825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98bc19f320 con 0x7f98bc1083b0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98baffd640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d750 tx=0x7f98b000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:23.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0017070 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98bc19f610 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98b00054d0 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98b0004c30 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.169+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98bc1a5d70 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.170+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98bc103cf0 con 0x7f98bc1083b0 2024-09-21T02:56:23.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.173+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f98b0004dd0 con 0x7f98bc1083b0 2024-09-21T02:56:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.174+0000 7f98b8ff9640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 0x7f98a40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.174+0000 7f98bb7fe640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 0x7f98a40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.174+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(53..53 src has 1..53) ==== 5764+0+0 (secure 0 0 0) 0x7f98b00e6690 con 0x7f98bc1083b0 2024-09-21T02:56:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.174+0000 7f98bb7fe640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 0x7f98a40b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f98ac031440 tx=0x7f98ac009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:23.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.177+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98b00b1cf0 con 0x7f98bc1083b0 2024-09-21T02:56:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:56:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:56:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:23.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T02:56:22.805+0000 7f55a7028640 -1 osd.8 0 waiting for initial osdmap 2024-09-21T02:56:23.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T02:56:22.816+0000 7f55a2e26640 -1 osd.8 53 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:56:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: pgmap v173: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: osdmap e52: 9 total, 8 up, 9 in 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743]' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd={"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:23.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:23.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:23.375+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f98bc063800 con 0x7f98a40b6560 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi123 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:24.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi123 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:24.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:24.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi123 2024-09-21T02:56:24.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi123 to 5696M 2024-09-21T02:56:24.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: osdmap e53: 9 total, 8 up, 9 in 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:24.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] boot 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:56:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] boot 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:56:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:56:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: pgmap v176: 1 pgs: 1 active+clean; 577 KiB data, 253 MiB used, 715 GiB / 715 GiB avail 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: from='client.14652 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi016:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: osd.8 [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] boot 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: osdmap e54: 9 total, 9 up, 9 in 2024-09-21T02:56:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 8} : dispatch 2024-09-21T02:56:26.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:25 smithi016 ceph-mon[31253]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:56:26.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:25 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"} : dispatch 2024-09-21T02:56:26.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:25 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"}]': finished 2024-09-21T02:56:26.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:25 smithi016 ceph-mon[31253]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:56:26.144 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:25 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:25 smithi018 ceph-mon[34587]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:56:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:25 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"} : dispatch 2024-09-21T02:56:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:25 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"}]': finished 2024-09-21T02:56:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:25 smithi018 ceph-mon[34587]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:56:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:25 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:26.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:25 smithi123 ceph-mon[34645]: osdmap e55: 9 total, 9 up, 9 in 2024-09-21T02:56:26.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:25 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"} : dispatch 2024-09-21T02:56:26.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:25 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2515608571' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "9d9f79be-daa6-4305-8037-4487eb7b3e5a"}]': finished 2024-09-21T02:56:26.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:25 smithi123 ceph-mon[34645]: osdmap e56: 10 total, 9 up, 10 in 2024-09-21T02:56:26.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:25 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:26 smithi018 ceph-mon[34587]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:26 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2555403727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:26 smithi123 ceph-mon[34645]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:26 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2555403727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:27.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:26 smithi016 ceph-mon[31253]: pgmap v179: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:27.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2555403727' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:28 smithi018 ceph-mon[34587]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:28 smithi123 ceph-mon[34645]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:29.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:28 smithi016 ceph-mon[31253]: pgmap v181: 1 pgs: 1 remapped+peering; 577 KiB data, 280 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:31.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:30 smithi016 ceph-mon[31253]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:30 smithi018 ceph-mon[34587]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:30 smithi123 ceph-mon[34645]: pgmap v182: 1 pgs: 1 remapped+peering; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:33.088 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:32 smithi016 ceph-mon[31253]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T02:56:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:32 smithi018 ceph-mon[34587]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T02:56:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:32 smithi123 ceph-mon[34645]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 79 KiB/s, 0 objects/s recovering 2024-09-21T02:56:35.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:34 smithi016 ceph-mon[31253]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T02:56:35.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:56:35.049 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:34 smithi018 ceph-mon[34587]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T02:56:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:56:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:34 smithi123 ceph-mon[34645]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 69 KiB/s, 0 objects/s recovering 2024-09-21T02:56:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.9"} : dispatch 2024-09-21T02:56:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:36.080 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:35 smithi016 ceph-mon[31253]: Deploying daemon osd.9 on smithi016 2024-09-21T02:56:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:35 smithi018 ceph-mon[34587]: Deploying daemon osd.9 on smithi016 2024-09-21T02:56:36.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:35 smithi123 ceph-mon[34645]: Deploying daemon osd.9 on smithi016 2024-09-21T02:56:37.122 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:36 smithi123 ceph-mon[34645]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:56:37.142 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:36 smithi016 ceph-mon[31253]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:56:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:36 smithi018 ceph-mon[34587]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-09-21T02:56:38.984 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:38 smithi016 ceph-mon[31253]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:56:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:38 smithi018 ceph-mon[34587]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:56:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:38 smithi123 ceph-mon[34645]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 51 KiB/s, 0 objects/s recovering 2024-09-21T02:56:40.900 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:40 smithi016 ceph-mon[31253]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:40.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:40 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:40.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:40 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:40.901 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:40 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:40 smithi018 ceph-mon[34587]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:40 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:40 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:40 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:40 smithi123 ceph-mon[34645]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:40 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:40 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:40 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:42 smithi018 ceph-mon[34587]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:42 smithi123 ceph-mon[34645]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:42 smithi016 ceph-mon[31253]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail; 48 KiB/s, 0 objects/s recovering 2024-09-21T02:56:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:43.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.557+0000 7f98b8ff9640 1 -- 172.21.15.16:0/1862391825 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f98bc063800 con 0x7f98a40b6560 2024-09-21T02:56:43.559 INFO:teuthology.orchestra.run.smithi016.stdout:Created osd(s) 9 on host 'smithi016' 2024-09-21T02:56:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 msgr2=0x7f98a40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 0x7f98a40b8a70 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f98ac031440 tx=0x7f98ac009560 comp rx=0 tx=0).stop 2024-09-21T02:56:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 msgr2=0x7f98bc199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f98b000d750 tx=0x7f98b000dc10 comp rx=0 tx=0).stop 2024-09-21T02:56:43.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 shutdown_connections 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98a40b6560 0x7f98a40b8a70 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98bc1083b0 0x7f98bc199950 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98bc108d00 0x7f98bc19ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 --2- 172.21.15.16:0/1862391825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98bc102a70 0x7f98bc1993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 >> 172.21.15.16:0/1862391825 conn(0x7f98bc079710 msgr2=0x7f98bc106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 shutdown_connections 2024-09-21T02:56:43.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:56:43.561+0000 7f98c1abf640 1 -- 172.21.15.16:0/1862391825 wait complete. 2024-09-21T02:56:44.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:44.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:44.103 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.104 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.131 DEBUG:teuthology.orchestra.run.smithi016:osd.9> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.9.service 2024-09-21T02:56:44.133 INFO:tasks.cephadm:Deploying osd.10 on smithi018 with /dev/vg_nvme/lv_1... 2024-09-21T02:56:44.133 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.161 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:44.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:44.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:56:44.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:44.397 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:56:45.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:44 smithi016 ceph-mon[31253]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:44 smithi018 ceph-mon[34587]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:44 smithi123 ceph-mon[34645]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:46.471 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 02:56:46 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T02:56:46.077+0000 7f56b6cef640 -1 osd.9 0 log_to_monitors true 2024-09-21T02:56:46.824 INFO:teuthology.orchestra.run.smithi018.stdout: 2024-09-21T02:56:46.850 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi018:vg_nvme/lv_1 2024-09-21T02:56:47.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:46 smithi016 ceph-mon[31253]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:47.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:46 smithi016 ceph-mon[31253]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:56:47.144 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T02:56:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:46 smithi018 ceph-mon[34587]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:46 smithi018 ceph-mon[34587]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:56:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:46 smithi123 ceph-mon[34645]: pgmap v190: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:46 smithi123 ceph-mon[34645]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]} : dispatch 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 -- 172.21.15.18:0/4262138073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 msgr2=0x7f7a8c10d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4262138073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c10d8b0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f7a80009900 tx=0x7f7a8002a410 comp rx=0 tx=0).stop 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 -- 172.21.15.18:0/4262138073 shutdown_connections 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4262138073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4262138073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c10d8b0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4262138073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 0x7f7a8c104c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 -- 172.21.15.18:0/4262138073 >> 172.21.15.18:0/4262138073 conn(0x7f7a8c079730 msgr2=0x7f7a8c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:56:47.624 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 -- 172.21.15.18:0/4262138073 shutdown_connections 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.609+0000 7f7a910b5640 1 -- 172.21.15.18:0/4262138073 wait complete. 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.610+0000 7f7a910b5640 1 Processor -- start 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.610+0000 7f7a910b5640 1 -- start start 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.610+0000 7f7a910b5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 0x7f7a8c102750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.610+0000 7f7a910b5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.610+0000 7f7a910b5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c1031d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a910b5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c113ab0 con 0x7f7a8c10ddf0 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a910b5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c113930 con 0x7f7a8c105270 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a910b5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a8c113c30 con 0x7f7a8c104840 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.18:49350/0 (socket says 172.21.15.18:49350) 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 -- 172.21.15.18:0/4008196124 learned_addr learned my addr 172.21.15.18:0/4008196124 (peer_addr_for_me v2:172.21.15.18:0/0) 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8ad9d640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 0x7f7a8c102750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.625 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8b59e640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c1031d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 -- 172.21.15.18:0/4008196124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 msgr2=0x7f7a8c102750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 0x7f7a8c102750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 -- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 msgr2=0x7f7a8c1031d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c1031d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 -- 172.21.15.18:0/4008196124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a8c1a8710 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8b59e640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a8a59c640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002a930 tx=0x7f7a80004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a8003f030 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a8c1a89a0 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a80003cb0 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.611+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a80035070 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.612+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a8c1a8f50 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.613+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a8c104c50 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.615+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f7a80042030 con 0x7f7a8c105270 2024-09-21T02:56:47.626 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.616+0000 7f7a73fff640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 0x7f7a740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:56:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.616+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(57..57 src has 1..57) ==== 6180+0+0 (secure 0 0 0) 0x7f7a80107610 con 0x7f7a8c105270 2024-09-21T02:56:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.616+0000 7f7a8ad9d640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 0x7f7a740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:56:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.616+0000 7f7a8ad9d640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 0x7f7a740b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c1041a0 tx=0x7f7a7c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:56:47.627 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.619+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a800ce650 con 0x7f7a8c105270 2024-09-21T02:56:47.819 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:56:47.817+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f7a8c063800 con 0x7f7a740b6560 2024-09-21T02:56:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:56:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:48.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:47 smithi018 ceph-mon[34587]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:48.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:48.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:48.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.222 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:47 smithi123 ceph-mon[34645]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:56:48.265 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T02:56:47.865+0000 7f56b2c71640 -1 osd.9 0 waiting for initial osdmap 2024-09-21T02:56:48.266 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T02:56:47.885+0000 7f56aea6f640 -1 osd.9 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: osdmap e57: 10 total, 9 up, 10 in 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd={"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]} : dispatch 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:48.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:47 smithi016 ceph-mon[31253]: from='osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi016", "root=default"]}]': finished 2024-09-21T02:56:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi016 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi016 2024-09-21T02:56:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T02:56:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:49.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:56:49.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: pgmap v192: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:49.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi016 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi016 to 4273M 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: from='client.24530 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi018:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: osdmap e58: 10 total, 9 up, 10 in 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:50.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:49 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:56:50.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:49 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:56:50.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:49 smithi018 ceph-mon[34587]: osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] boot 2024-09-21T02:56:50.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:49 smithi018 ceph-mon[34587]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:56:50.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:49 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:49 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:56:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:49 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:56:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:49 smithi123 ceph-mon[34645]: osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] boot 2024-09-21T02:56:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:49 smithi123 ceph-mon[34645]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:56:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:49 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:50.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:49 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:56:50.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:49 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:56:50.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:49 smithi016 ceph-mon[31253]: osd.9 [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] boot 2024-09-21T02:56:50.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:49 smithi016 ceph-mon[31253]: osdmap e59: 10 total, 10 up, 10 in 2024-09-21T02:56:50.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:49 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 9} : dispatch 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/3380137950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"}]': finished 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:56:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:50 smithi018 ceph-mon[34587]: from='client.? 172.21.15.18:0/2664500800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/3380137950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"}]': finished 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:56:51.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:50 smithi123 ceph-mon[34645]: from='client.? 172.21.15.18:0/2664500800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: pgmap v195: 1 pgs: 1 active+clean; 577 KiB data, 281 MiB used, 804 GiB / 805 GiB avail 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: osdmap e60: 10 total, 10 up, 10 in 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/3380137950' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"} : dispatch 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "84176f3a-eba3-4fa1-aaf6-523a10b9d713"}]': finished 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: osdmap e61: 11 total, 10 up, 11 in 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:56:51.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:50 smithi016 ceph-mon[31253]: from='client.? 172.21.15.18:0/2664500800' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:56:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:52 smithi018 ceph-mon[34587]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:52 smithi123 ceph-mon[34645]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:53.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:52 smithi016 ceph-mon[31253]: pgmap v198: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:54 smithi018 ceph-mon[34587]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:54 smithi123 ceph-mon[34645]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:55.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:54 smithi016 ceph-mon[31253]: pgmap v199: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:56 smithi018 ceph-mon[34587]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:56 smithi123 ceph-mon[34645]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:57.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:56 smithi016 ceph-mon[31253]: pgmap v200: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:58 smithi018 ceph-mon[34587]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:56:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:58 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:58 smithi123 ceph-mon[34645]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:56:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:58 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:56:59.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:58 smithi016 ceph-mon[31253]: pgmap v201: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:56:59.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.10"} : dispatch 2024-09-21T02:56:59.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:58 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:00.079 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:56:59 smithi018 ceph-mon[34587]: Deploying daemon osd.10 on smithi018 2024-09-21T02:57:00.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:56:59 smithi123 ceph-mon[34645]: Deploying daemon osd.10 on smithi018 2024-09-21T02:57:00.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:56:59 smithi016 ceph-mon[31253]: Deploying daemon osd.10 on smithi018 2024-09-21T02:57:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:00 smithi018 ceph-mon[34587]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:00 smithi123 ceph-mon[34645]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:01.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:00 smithi016 ceph-mon[31253]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:01.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:01 smithi018 ceph-mon[34587]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:02.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:01 smithi123 ceph-mon[34645]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:02.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:01 smithi016 ceph-mon[31253]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:04.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:04 smithi018 ceph-mon[34587]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:04.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:04.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:04.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:04.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:04 smithi123 ceph-mon[34645]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:04.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:04.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:04.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:04.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:04 smithi016 ceph-mon[31253]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:04.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:04.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:04.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:06.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:06 smithi018 ceph-mon[34587]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:06.451 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:06 smithi016 ceph-mon[31253]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:06.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:06 smithi123 ceph-mon[34645]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:06.984 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.983+0000 7f7a73fff640 1 -- 172.21.15.18:0/4008196124 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f7a8c063800 con 0x7f7a740b6560 2024-09-21T02:57:06.984 INFO:teuthology.orchestra.run.smithi018.stdout:Created osd(s) 10 on host 'smithi018' 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 msgr2=0x7f7a740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 0x7f7a740b8a70 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c1041a0 tx=0x7f7a7c005f90 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 msgr2=0x7f7a8c102c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7a8002a930 tx=0x7f7a80004270 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 shutdown_connections 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.985+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a740b6560 0x7f7a740b8a70 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a8c10ddf0 0x7f7a8c1031d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a8c105270 0x7f7a8c102c90 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 --2- 172.21.15.18:0/4008196124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a8c104840 0x7f7a8c102750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:06.987 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 >> 172.21.15.18:0/4008196124 conn(0x7f7a8c079730 msgr2=0x7f7a8c100880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:06.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 shutdown_connections 2024-09-21T02:57:06.988 INFO:teuthology.orchestra.run.smithi018.stderr:2024-09-21T02:57:06.986+0000 7f7a910b5640 1 -- 172.21.15.18:0/4008196124 wait complete. 2024-09-21T02:57:07.997 DEBUG:teuthology.orchestra.run.smithi018:osd.10> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.10.service 2024-09-21T02:57:07.998 INFO:tasks.cephadm:Deploying osd.11 on smithi123 with /dev/vg_nvme/lv_1... 2024-09-21T02:57:07.999 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 ceph-volume -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- lvm zap /dev/vg_nvme/lv_1 2024-09-21T02:57:08.253 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:57:08.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:07 smithi016 ceph-mon[31253]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:08.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:07 smithi123 ceph-mon[34645]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:08.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:07 smithi018 ceph-mon[34587]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:09.380 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:09.380 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:09 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:09.426 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:09.426 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:09 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:09.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:09.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:09 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:10.237 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:10 smithi123 ceph-mon[34645]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:10 smithi018 ceph-mon[34587]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:10.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 02:57:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T02:57:10.173+0000 7f6ef4e83640 -1 osd.10 0 log_to_monitors true 2024-09-21T02:57:10.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:10 smithi016 ceph-mon[31253]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:10.829 INFO:teuthology.orchestra.run.smithi123.stdout: 2024-09-21T02:57:10.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch daemon add osd smithi123:vg_nvme/lv_1 2024-09-21T02:57:11.113 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:11 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:11.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:11 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:11.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:11 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 -- 172.21.15.123:0/4073400315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b0102f40 msgr2=0x7f01b010fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/4073400315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b0102f40 0x7f01b010fc60 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f01a4009900 tx=0x7f01a402a480 comp rx=0 tx=0).stop 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 -- 172.21.15.123:0/4073400315 shutdown_connections 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/4073400315 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b01101a0 0x7f01b01125c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/4073400315 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b0102f40 0x7f01b010fc60 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/4073400315 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 0x7f01b0102a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 -- 172.21.15.123:0/4073400315 >> 172.21.15.123:0/4073400315 conn(0x7f01b0079740 msgr2=0x7f01b0079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 -- 172.21.15.123:0/4073400315 shutdown_connections 2024-09-21T02:57:11.623 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.621+0000 7f01b7bc5640 1 -- 172.21.15.123:0/4073400315 wait complete. 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 Processor -- start 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 -- start start 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 0x7f01b019c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 0x7f01b019cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:11.625 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0115ca0 con 0x7f01b01025f0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0115b20 con 0x7f01b0102f40 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.622+0000 7f01b7bc5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01b0115e20 con 0x7f01b01101a0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41424/0 (socket says 172.21.15.123:41424) 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 -- 172.21.15.123:0/811020777 learned_addr learned my addr 172.21.15.123:0/811020777 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b5160640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 0x7f01b019cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b5961640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 0x7f01b019c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 -- 172.21.15.123:0/811020777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 msgr2=0x7f01b019cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 0x7f01b019cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 -- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 msgr2=0x7f01b019c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 0x7f01b019c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 -- 172.21.15.123:0/811020777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01b01a2600 con 0x7f01b01101a0 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b5160640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 0x7f01b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:11.626 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b6162640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01a800d750 tx=0x7f01a800dc10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8017070 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01b01a28f0 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01a80054d0 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01a8004c60 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.623+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01b01a9050 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.625+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01b006c240 con 0x7f01b01101a0 2024-09-21T02:57:11.627 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.626+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f01a8009bd0 con 0x7f01b01101a0 2024-09-21T02:57:11.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.627+0000 7f01aeffd640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 0x7f01980b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:11.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.627+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 6613+0+0 (secure 0 0 0) 0x7f01a80e66d0 con 0x7f01b01101a0 2024-09-21T02:57:11.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.627+0000 7f01b5961640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 0x7f01980b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:11.628 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.627+0000 7f01b5961640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 0x7f01980b8a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005c40 tx=0x7f01a000a780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:11.632 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.631+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01a80ad600 con 0x7f01b01101a0 2024-09-21T02:57:11.818 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:11.817+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f01b010a450 con 0x7f01980b6560 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi018 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi018 to 4273M 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi018 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi018 to 4273M 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:57:12.433 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:57:12.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:12 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:12.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi018 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi018 to 4273M 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: osdmap e62: 11 total, 10 up, 11 in 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437]' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='osd.10 ' entity='osd.10' cmd={"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]} : dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='client.24538 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi123:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd tree", "states": ["destroyed"], "format": "json"} : dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.bootstrap-osd"} : dispatch 2024-09-21T02:57:12.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:12 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:13.180 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T02:57:12.893+0000 7f6ef0e05640 -1 osd.10 0 waiting for initial osdmap 2024-09-21T02:57:13.180 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 02:57:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T02:57:12.906+0000 7f6ee8bfb640 -1 osd.10 63 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:13 smithi018 ceph-mon[34587]: from='osd.10 ' entity='osd.10' 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.437 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:13 smithi123 ceph-mon[34645]: from='osd.10 ' entity='osd.10' 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi018", "root=default"]}]': finished 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: osdmap e63: 11 total, 10 up, 11 in 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:13.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:13 smithi016 ceph-mon[31253]: from='osd.10 ' entity='osd.10' 2024-09-21T02:57:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:14 smithi018 ceph-mon[34587]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:14 smithi018 ceph-mon[34587]: osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] boot 2024-09-21T02:57:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:14 smithi018 ceph-mon[34587]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T02:57:14.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:14 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:14.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:14 smithi123 ceph-mon[34645]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:14.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:14.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:14 smithi123 ceph-mon[34645]: osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] boot 2024-09-21T02:57:14.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:14 smithi123 ceph-mon[34645]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T02:57:14.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:14 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:14.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:14 smithi016 ceph-mon[31253]: pgmap v211: 1 pgs: 1 active+clean; 577 KiB data, 308 MiB used, 894 GiB / 894 GiB avail 2024-09-21T02:57:14.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:14.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:14 smithi016 ceph-mon[31253]: osd.10 [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] boot 2024-09-21T02:57:14.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:14 smithi016 ceph-mon[31253]: osdmap e64: 11 total, 11 up, 11 in 2024-09-21T02:57:14.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:14 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 10} : dispatch 2024-09-21T02:57:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:57:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/3011221143' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"}]': finished 2024-09-21T02:57:15.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:57:15.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:15.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:15 smithi123 ceph-mon[34645]: from='client.? 172.21.15.123:0/214657940' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:57:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:57:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/3011221143' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"}]': finished 2024-09-21T02:57:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:57:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:15.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.123:0/214657940' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: osdmap e65: 11 total, 11 up, 11 in 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/3011221143' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd={"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"} : dispatch 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "340835c3-2eab-478a-9def-47542b5fc7bc"}]': finished 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: osdmap e66: 12 total, 11 up, 12 in 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:15.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:15 smithi018 ceph-mon[34587]: from='client.? 172.21.15.123:0/214657940' entity='client.bootstrap-osd' cmd={"prefix": "mon getmap"} : dispatch 2024-09-21T02:57:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:16 smithi018 ceph-mon[34587]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-21T02:57:16.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:16 smithi123 ceph-mon[34645]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-21T02:57:16.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:16 smithi016 ceph-mon[31253]: pgmap v215: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 146 KiB/s, 0 objects/s recovering 2024-09-21T02:57:18.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:18 smithi018 ceph-mon[34587]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-21T02:57:18.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:18 smithi123 ceph-mon[34645]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-21T02:57:18.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:18 smithi016 ceph-mon[31253]: pgmap v216: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-09-21T02:57:20.429 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:20 smithi123 ceph-mon[34645]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:57:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:20 smithi018 ceph-mon[34587]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:57:20.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:20 smithi016 ceph-mon[31253]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-09-21T02:57:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:22 smithi018 ceph-mon[34587]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T02:57:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:22 smithi123 ceph-mon[34645]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T02:57:22.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:22 smithi016 ceph-mon[31253]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 73 KiB/s, 0 objects/s recovering 2024-09-21T02:57:23.619 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:57:23.620 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:23.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:57:23.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "osd.11"} : dispatch 2024-09-21T02:57:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:24.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:24 smithi123 ceph-mon[34645]: Deploying daemon osd.11 on smithi123 2024-09-21T02:57:24.679 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:24 smithi123 ceph-mon[34645]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T02:57:24.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:24 smithi018 ceph-mon[34587]: Deploying daemon osd.11 on smithi123 2024-09-21T02:57:24.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:24 smithi018 ceph-mon[34587]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T02:57:24.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:24 smithi016 ceph-mon[31253]: Deploying daemon osd.11 on smithi123 2024-09-21T02:57:24.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:24 smithi016 ceph-mon[31253]: pgmap v219: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-09-21T02:57:26.584 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:26 smithi123 ceph-mon[34645]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T02:57:26.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:26 smithi018 ceph-mon[34587]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T02:57:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:26 smithi016 ceph-mon[31253]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail; 53 KiB/s, 0 objects/s recovering 2024-09-21T02:57:28.461 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:28 smithi123 ceph-mon[34645]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:28 smithi018 ceph-mon[34587]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:28.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:28 smithi016 ceph-mon[31253]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:29.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:29.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:29.411 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:29 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:29.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:29 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:29.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:29 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:29.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:29 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:29.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:29.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:29.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:29 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:31.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:30 smithi016 ceph-mon[31253]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:31.123 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:30 smithi123 ceph-mon[34645]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:30 smithi018 ceph-mon[34587]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:32.216 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.213+0000 7f01aeffd640 1 -- 172.21.15.123:0/811020777 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f01b010a450 con 0x7f01980b6560 2024-09-21T02:57:32.216 INFO:teuthology.orchestra.run.smithi123.stdout:Created osd(s) 11 on host 'smithi123' 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 msgr2=0x7f01980b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 0x7f01980b8a70 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01a0005c40 tx=0x7f01a000a780 comp rx=0 tx=0).stop 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 msgr2=0x7f01b01a1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f01a800d750 tx=0x7f01a800dc10 comp rx=0 tx=0).stop 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 shutdown_connections 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01980b6560 0x7f01980b8a70 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01b01101a0 0x7f01b01a1ed0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:32.218 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01b0102f40 0x7f01b019cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:32.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 --2- 172.21.15.123:0/811020777 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01b01025f0 0x7f01b019c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:32.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 >> 172.21.15.123:0/811020777 conn(0x7f01b0079740 msgr2=0x7f01b0109d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:32.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 shutdown_connections 2024-09-21T02:57:32.219 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:32.216+0000 7f01b7bc5640 1 -- 172.21.15.123:0/811020777 wait complete. 2024-09-21T02:57:32.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:32 smithi123 ceph-mon[34645]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:32.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:32 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:32.829 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:32 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:32.858 DEBUG:teuthology.orchestra.run.smithi123:osd.11> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.11.service 2024-09-21T02:57:32.860 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-09-21T02:57:32.860 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:57:33.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:32 smithi016 ceph-mon[31253]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:33.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:32 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:33.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:32 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:33.136 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:32 smithi018 ceph-mon[34587]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:32 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:32 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:33.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/1048741422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4109e00 msgr2=0x7fb6a4110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/1048741422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4109e00 0x7fb6a4110630 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fb69c009ca0 tx=0x7fb69c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/1048741422 shutdown_connections 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/1048741422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4109e00 0x7fb6a4110630 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/1048741422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a41094b0 0x7fb6a41098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/1048741422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a4101f70 0x7fb6a4102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/1048741422 >> 172.21.15.16:0/1048741422 conn(0x7fb6a4077c60 msgr2=0x7fb6a40780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/1048741422 shutdown_connections 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.607+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/1048741422 wait complete. 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.608+0000 7fb6aa1a0640 1 Processor -- start 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.608+0000 7fb6aa1a0640 1 -- start start 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 0x7fb6a419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a4109e00 0x7fb6a419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4113ac0 con 0x7fb6a4101f70 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4113940 con 0x7fb6a4109e00 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6aa1a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6a4113c40 con 0x7fb6a41094b0 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57950/0 (socket says 172.21.15.16:57950) 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 -- 172.21.15.16:0/2821754842 learned_addr learned my addr 172.21.15.16:0/2821754842 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a2ffd640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 0x7fb6a419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a3fff640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a4109e00 0x7fb6a419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 -- 172.21.15.16:0/2821754842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 msgr2=0x7fb6a419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 0x7fb6a419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 -- 172.21.15.16:0/2821754842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a4109e00 msgr2=0x7fb6a419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a4109e00 0x7fb6a419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 -- 172.21.15.16:0/2821754842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6a41a0460 con 0x7fb6a4101f70 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a2ffd640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 0x7fb6a419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:57:33.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.609+0000 7fb6a37fe640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb69400ea60 tx=0x7fb69400ef20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.610+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb69400cde0 con 0x7fb6a4101f70 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.610+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6a41a0750 con 0x7fb6a4101f70 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.610+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6940035f0 con 0x7fb6a4101f70 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.610+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6a41a6eb0 con 0x7fb6a4101f70 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.610+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb694010690 con 0x7fb6a4101f70 2024-09-21T02:57:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.611+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6a4102380 con 0x7fb6a4101f70 2024-09-21T02:57:33.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.613+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb6940037a0 con 0x7fb6a4101f70 2024-09-21T02:57:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.613+0000 7fb6a0ff9640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 0x7fb68c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.614+0000 7fb6a2ffd640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 0x7fb68c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.614+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(66..66 src has 1..66) ==== 7024+0+0 (secure 0 0 0) 0x7fb6940e74b0 con 0x7fb6a4101f70 2024-09-21T02:57:33.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.614+0000 7fb6a2ffd640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 0x7fb68c0b8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4109130 tx=0x7fb69800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.617+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb6940ae250 con 0x7fb6a4101f70 2024-09-21T02:57:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.806+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fb6a410bc50 con 0x7fb6a4101f70 2024-09-21T02:57:33.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.806+0000 7fb6a0ff9640 1 -- 172.21.15.16:0/2821754842 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v66) ==== 74+0+133 (secure 0 0 0) 0x7fb6940b2630 con 0x7fb6a4101f70 2024-09-21T02:57:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.808+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 msgr2=0x7fb68c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 0x7fb68c0b8af0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7fb6a4109130 tx=0x7fb69800b410 comp rx=0 tx=0).stop 2024-09-21T02:57:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 msgr2=0x7fb6a419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fb69400ea60 tx=0x7fb69400ef20 comp rx=0 tx=0).stop 2024-09-21T02:57:33.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 shutdown_connections 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb68c0b65e0 0x7fb68c0b8af0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6a4109e00 0x7fb6a419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6a41094b0 0x7fb6a419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 --2- 172.21.15.16:0/2821754842 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb6a4101f70 0x7fb6a419a4f0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 >> 172.21.15.16:0/2821754842 conn(0x7fb6a4077c60 msgr2=0x7fb6a4107540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 shutdown_connections 2024-09-21T02:57:33.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:33.809+0000 7fb6aa1a0640 1 -- 172.21.15.16:0/2821754842 wait complete. 2024-09-21T02:57:34.371 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.371 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.371 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:34 smithi018 ceph-mon[34587]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:34.371 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:34 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2821754842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:34.426 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.426 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.427 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:34 smithi123 ceph-mon[34645]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:34.427 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:34 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2821754842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:34.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:34.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:34 smithi016 ceph-mon[31253]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:34.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:34 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2821754842' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:34.706 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":66,"num_osds":12,"num_up_osds":11,"osd_up_since":1726887433,"num_in_osds":12,"osd_in_since":1726887434,"num_remapped_pgs":0} 2024-09-21T02:57:35.353 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:35 smithi018 ceph-mon[34587]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.354 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:35 smithi018 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.449 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:35 smithi123 ceph-mon[34645]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.449 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:35 smithi123 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.449 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 02:57:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T02:57:35.060+0000 7f05ebefd640 -1 osd.11 0 log_to_monitors true 2024-09-21T02:57:35.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:35 smithi016 ceph-mon[31253]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:35 smithi016 ceph-mon[31253]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]} : dispatch 2024-09-21T02:57:35.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:57:35.977 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:36.220 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T02:57:36.082+0000 7f05e7e7f640 -1 osd.11 0 waiting for initial osdmap 2024-09-21T02:57:36.220 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T02:57:36.098+0000 7f05e3c7d640 -1 osd.11 68 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:36 smithi123 ceph-mon[34645]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.348 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:36 smithi016 ceph-mon[31253]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: osdmap e67: 12 total, 11 up, 12 in 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: from='osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931]' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd={"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]} : dispatch 2024-09-21T02:57:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:36 smithi018 ceph-mon[34587]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 344 MiB used, 983 GiB / 983 GiB avail 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 -- 172.21.15.16:0/182040292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe41083c0 msgr2=0x7fbbe41087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 --2- 172.21.15.16:0/182040292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe41083c0 0x7fbbe41087d0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009900 tx=0x7fbbd802a410 comp rx=0 tx=0).stop 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 -- 172.21.15.16:0/182040292 shutdown_connections 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 --2- 172.21.15.16:0/182040292 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe41083c0 0x7fbbe41087d0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 --2- 172.21.15.16:0/182040292 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe4108d10 0x7fbbe410f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 --2- 172.21.15.16:0/182040292 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4102a80 0x7fbbe4102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 -- 172.21.15.16:0/182040292 >> 172.21.15.16:0/182040292 conn(0x7fbbe4079740 msgr2=0x7fbbe4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 -- 172.21.15.16:0/182040292 shutdown_connections 2024-09-21T02:57:36.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.439+0000 7fbbe994e640 1 -- 172.21.15.16:0/182040292 wait complete. 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.440+0000 7fbbe994e640 1 Processor -- start 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 -- start start 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe41083c0 0x7fbbe4198830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4108d10 0x7fbbe419dad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4111880 con 0x7fbbe4102a80 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4111700 con 0x7fbbe41083c0 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe994e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbbe4111a00 con 0x7fbbe4108d10 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58766/0 (socket says 172.21.15.16:58766) 2024-09-21T02:57:36.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe2ffd640 1 -- 172.21.15.16:0/4199417153 learned_addr learned my addr 172.21.15.16:0/4199417153 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe37fe640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4108d10 0x7fbbe419dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.441+0000 7fbbe27fc640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe41083c0 0x7fbbe4198830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 -- 172.21.15.16:0/4199417153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4108d10 msgr2=0x7fbbe419dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4108d10 0x7fbbe419dad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 -- 172.21.15.16:0/4199417153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe41083c0 msgr2=0x7fbbe4198830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe41083c0 0x7fbbe4198830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 -- 172.21.15.16:0/4199417153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbe419e200 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe2ffd640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e9f0 tx=0x7fbbd400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd400ccc0 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbe419e4f0 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbd400ce60 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbd4010430 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.442+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbe41a4c50 con 0x7fbbe4102a80 2024-09-21T02:57:36.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.443+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbe4103d00 con 0x7fbbe4102a80 2024-09-21T02:57:36.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.446+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbbd40046c0 con 0x7fbbe4102a80 2024-09-21T02:57:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.446+0000 7fbbe894c640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 0x7fbbcc0bac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.446+0000 7fbbe27fc640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 0x7fbbcc0bac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.447+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 7062+0+0 (secure 0 0 0) 0x7fbbd40e7440 con 0x7fbbe4102a80 2024-09-21T02:57:36.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.447+0000 7fbbe27fc640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 0x7fbbcc0bac80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009230 tx=0x7fbbd8005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:36.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.450+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbbd4056710 con 0x7fbbe4102a80 2024-09-21T02:57:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.639+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7fbbe4102e90 con 0x7fbbe4102a80 2024-09-21T02:57:36.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.640+0000 7fbbe894c640 1 -- 172.21.15.16:0/4199417153 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v68) ==== 74+0+133 (secure 0 0 0) 0x7fbbe4102e90 con 0x7fbbe4102a80 2024-09-21T02:57:36.641 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 msgr2=0x7fbbcc0bac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 0x7fbbcc0bac80 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fbbd8009230 tx=0x7fbbd8005a20 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 msgr2=0x7fbbe4198290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fbbd400e9f0 tx=0x7fbbd400edf0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 shutdown_connections 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbbcc0b8770 0x7fbbcc0bac80 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbbe4108d10 0x7fbbe419dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbbe41083c0 0x7fbbe4198830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 --2- 172.21.15.16:0/4199417153 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbbe4102a80 0x7fbbe4198290 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.642+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 >> 172.21.15.16:0/4199417153 conn(0x7fbbe4079740 msgr2=0x7fbbe4104490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:36.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.643+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 shutdown_connections 2024-09-21T02:57:36.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:36.643+0000 7fbbe994e640 1 -- 172.21.15.16:0/4199417153 wait complete. 2024-09-21T02:57:37.289 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":68,"num_osds":12,"num_up_osds":11,"osd_up_since":1726887433,"num_in_osds":12,"osd_in_since":1726887434,"num_remapped_pgs":0} 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: Detected new or changed devices on smithi123 2024-09-21T02:57:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:37 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4199417153' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:37.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:57:37.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: Detected new or changed devices on smithi123 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:37.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:37 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4199417153' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:37.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi123", "root=default"]}]': finished 2024-09-21T02:57:37.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: osdmap e68: 12 total, 11 up, 12 in 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: Detected new or changed devices on smithi123 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi123 to 4272M 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:37.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:37.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:37.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:37.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:37 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4199417153' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:38.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd stat -f json 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: purged_snaps scrub starts 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: purged_snaps scrub ok 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] boot 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:38.372 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:38 smithi123 ceph-mon[34645]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: purged_snaps scrub starts 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: purged_snaps scrub ok 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] boot 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:38 smithi018 ceph-mon[34587]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: purged_snaps scrub starts 2024-09-21T02:57:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: purged_snaps scrub ok 2024-09-21T02:57:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: osd.11 [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] boot 2024-09-21T02:57:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: osdmap e69: 12 total, 12 up, 12 in 2024-09-21T02:57:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd metadata", "id": 11} : dispatch 2024-09-21T02:57:38.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:38 smithi016 ceph-mon[31253]: pgmap v229: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:38.560 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:39.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 -- 172.21.15.16:0/626451200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c10de60 msgr2=0x7f368c110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 --2- 172.21.15.16:0/626451200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c10de60 0x7f368c110330 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f3684009ca0 tx=0x7f368402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 -- 172.21.15.16:0/626451200 shutdown_connections 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 --2- 172.21.15.16:0/626451200 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c10de60 0x7f368c110330 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 --2- 172.21.15.16:0/626451200 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 --2- 172.21.15.16:0/626451200 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c104a30 0x7f368c104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 -- 172.21.15.16:0/626451200 >> 172.21.15.16:0/626451200 conn(0x7f368c079760 msgr2=0x7f368c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 -- 172.21.15.16:0/626451200 shutdown_connections 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.021+0000 7f3690f16640 1 -- 172.21.15.16:0/626451200 wait complete. 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 Processor -- start 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 -- start start 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c10de60 0x7f368c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f368c113aa0 con 0x7f368c104a30 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f368c113920 con 0x7f368c105380 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3690f16640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f368c113c20 con 0x7f368c10de60 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58784/0 (socket says 172.21.15.16:58784) 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 -- 172.21.15.16:0/2149491290 learned_addr learned my addr 172.21.15.16:0/2149491290 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368ad9d640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c10de60 0x7f368c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3689d9b640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 -- 172.21.15.16:0/2149491290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c10de60 msgr2=0x7f368c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c10de60 0x7f368c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 -- 172.21.15.16:0/2149491290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 msgr2=0x7f368c19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f368a59c640 1 -- 172.21.15.16:0/2149491290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f368c1a0380 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.022+0000 7f3689d9b640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f368a59c640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f367c00d950 tx=0x7f367c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c017020 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f368c1a0670 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f367c009c30 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f367c00f690 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.023+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f368c1a6dd0 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.024+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f368c06c1f0 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.026+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f367c01a020 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.027+0000 7f367b7fe640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 0x7f36740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.027+0000 7f3689d9b640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 0x7f36740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.027+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f367c0e72d0 con 0x7f368c104a30 2024-09-21T02:57:39.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.028+0000 7f3689d9b640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 0x7f36740b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f36800094d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:39.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.030+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f367c021020 con 0x7f368c104a30 2024-09-21T02:57:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.219+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f368c104e40 con 0x7f368c104a30 2024-09-21T02:57:39.221 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:39 smithi016 ceph-mon[31253]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:57:39.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.220+0000 7f367b7fe640 1 -- 172.21.15.16:0/2149491290 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v70) ==== 74+0+133 (secure 0 0 0) 0x7f367c009de0 con 0x7f368c104a30 2024-09-21T02:57:39.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 msgr2=0x7f36740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 0x7f36740b8a70 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3680005fd0 tx=0x7f36800094d0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 msgr2=0x7f368c19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f367c00d950 tx=0x7f367c00de10 comp rx=0 tx=0).stop 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 shutdown_connections 2024-09-21T02:57:39.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36740b6560 0x7f36740b8a70 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f368c10de60 0x7f368c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f368c105380 0x7f368c19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 --2- 172.21.15.16:0/2149491290 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f368c104a30 0x7f368c19a410 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.222+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 >> 172.21.15.16:0/2149491290 conn(0x7f368c079760 msgr2=0x7f368c105850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.223+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 shutdown_connections 2024-09-21T02:57:39.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:39.223+0000 7f3690f16640 1 -- 172.21.15.16:0/2149491290 wait complete. 2024-09-21T02:57:39.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:39 smithi018 ceph-mon[34587]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:57:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:39 smithi123 ceph-mon[34645]: osdmap e70: 12 total, 12 up, 12 in 2024-09-21T02:57:39.947 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"num_osds":12,"num_up_osds":12,"osd_up_since":1726887457,"num_in_osds":12,"osd_in_since":1726887434,"num_remapped_pgs":0} 2024-09-21T02:57:39.947 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:57:40.209 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:40.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:40 smithi016 ceph-mon[31253]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:40.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:40 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2149491290' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:40.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:40 smithi018 ceph-mon[34587]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:40.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:40 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2149491290' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:40.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:40 smithi123 ceph-mon[34645]: pgmap v231: 1 pgs: 1 active+clean; 577 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:40.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:40 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2149491290' entity='client.admin' cmd={"prefix": "osd stat", "format": "json"} : dispatch 2024-09-21T02:57:40.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 -- 172.21.15.16:0/1519666417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 msgr2=0x7f76a8110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:40.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 --2- 172.21.15.16:0/1519666417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a8110620 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7698009ca0 tx=0x7f769802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 -- 172.21.15.16:0/1519666417 shutdown_connections 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 --2- 172.21.15.16:0/1519666417 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a8110620 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 --2- 172.21.15.16:0/1519666417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a81094a0 0x7f76a81098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 --2- 172.21.15.16:0/1519666417 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a8103b60 0x7f76a8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 -- 172.21.15.16:0/1519666417 >> 172.21.15.16:0/1519666417 conn(0x7f76a807b850 msgr2=0x7f76a807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 -- 172.21.15.16:0/1519666417 shutdown_connections 2024-09-21T02:57:40.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 -- 172.21.15.16:0/1519666417 wait complete. 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.663+0000 7f76b047f640 1 Processor -- start 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 -- start start 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a8103b60 0x7f76a819a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a81094a0 0x7f76a819aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113ab0 con 0x7f76a8109df0 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113930 con 0x7f76a81094a0 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76b047f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76a8113c30 con 0x7f76a8103b60 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58808/0 (socket says 172.21.15.16:58808) 2024-09-21T02:57:40.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 -- 172.21.15.16:0/2293044834 learned_addr learned my addr 172.21.15.16:0/2293044834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76ae21b640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a8103b60 0x7f76a819a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76ada1a640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a81094a0 0x7f76a819aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 -- 172.21.15.16:0/2293044834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a8103b60 msgr2=0x7f76a819a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a8103b60 0x7f76a819a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 -- 172.21.15.16:0/2293044834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a81094a0 msgr2=0x7f76a819aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a81094a0 0x7f76a819aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.664+0000 7f76aea1c640 1 -- 172.21.15.16:0/2293044834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76a81a03d0 con 0x7f76a8109df0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76aea1c640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7698005cc0 tx=0x7f769802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7698003bf0 con 0x7f76a8109df0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7698003d90 con 0x7f76a8109df0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76a81a0660 con 0x7f76a8109df0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f769802fbc0 con 0x7f76a8109df0 2024-09-21T02:57:40.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.665+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76a81a6e20 con 0x7f76a8109df0 2024-09-21T02:57:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.666+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76a8103f70 con 0x7f76a8109df0 2024-09-21T02:57:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.667+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f769802fd60 con 0x7f76a8109df0 2024-09-21T02:57:40.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.668+0000 7f76a77fe640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 0x7f76900b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:40.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.668+0000 7f76ae21b640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 0x7f76900b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:40.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.668+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f7698108400 con 0x7f76a8109df0 2024-09-21T02:57:40.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.669+0000 7f76ae21b640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 0x7f76900b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005d00 tx=0x7f76a000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:40.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.673+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76980cefc0 con 0x7f76a8109df0 2024-09-21T02:57:40.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.861+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f76a810b650 con 0x7f76a8109df0 2024-09-21T02:57:40.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.861+0000 7f76a77fe640 1 -- 172.21.15.16:0/2293044834 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v70) ==== 74+0+18771 (secure 0 0 0) 0x7f76980098b0 con 0x7f76a8109df0 2024-09-21T02:57:40.863 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:40.863 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":70,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:57:38.086755+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":39,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":48,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":65,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:57:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 msgr2=0x7f76900b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 0x7f76900b8af0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f76a0005d00 tx=0x7f76a000b410 comp rx=0 tx=0).stop 2024-09-21T02:57:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 msgr2=0x7f76a819fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f7698005cc0 tx=0x7f769802f380 comp rx=0 tx=0).stop 2024-09-21T02:57:40.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 shutdown_connections 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76900b65e0 0x7f76900b8af0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76a8109df0 0x7f76a819fca0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76a81094a0 0x7f76a819aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 --2- 172.21.15.16:0/2293044834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76a8103b60 0x7f76a819a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 >> 172.21.15.16:0/2293044834 conn(0x7f76a807b850 msgr2=0x7f76a81074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 shutdown_connections 2024-09-21T02:57:40.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:40.864+0000 7f76b047f640 1 -- 172.21.15.16:0/2293044834 wait complete. 2024-09-21T02:57:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:41 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2293044834' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:57:41.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:41 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2293044834' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:57:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:41 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2293044834' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:57:41.687 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:53:55.521347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T02:57:41.687 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd pool get .mgr pg_num 2024-09-21T02:57:41.948 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:42.363 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:42 smithi016 ceph-mon[31253]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 -- 172.21.15.16:0/599838525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9541083d0 msgr2=0x7fb9541087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 --2- 172.21.15.16:0/599838525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9541083d0 0x7fb9541087e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb948009900 tx=0x7fb94802a410 comp rx=0 tx=0).stop 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 -- 172.21.15.16:0/599838525 shutdown_connections 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 --2- 172.21.15.16:0/599838525 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb9541083d0 0x7fb9541087e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 --2- 172.21.15.16:0/599838525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 0x7fb95410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 --2- 172.21.15.16:0/599838525 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb954102ae0 0x7fb954102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 -- 172.21.15.16:0/599838525 >> 172.21.15.16:0/599838525 conn(0x7fb954079760 msgr2=0x7fb954079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 -- 172.21.15.16:0/599838525 shutdown_connections 2024-09-21T02:57:42.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.381+0000 7fb95a425640 1 -- 172.21.15.16:0/599838525 wait complete. 2024-09-21T02:57:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.382+0000 7fb95a425640 1 Processor -- start 2024-09-21T02:57:42.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.382+0000 7fb95a425640 1 -- start start 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9541083d0 0x7fb95419aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 0x7fb95419fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb954113ad0 con 0x7fb954102ae0 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb954113950 con 0x7fb9541083d0 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb95a425640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb954113c50 con 0x7fb954108d20 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58826/0 (socket says 172.21.15.16:58826) 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 -- 172.21.15.16:0/937178722 learned_addr learned my addr 172.21.15.16:0/937178722 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb9537fe640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9541083d0 0x7fb95419aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb9589c2640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 0x7fb95419fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 -- 172.21.15.16:0/937178722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 msgr2=0x7fb95419fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 0x7fb95419fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 -- 172.21.15.16:0/937178722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9541083d0 msgr2=0x7fb95419aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9541083d0 0x7fb95419aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 -- 172.21.15.16:0/937178722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9541a0430 con 0x7fb954102ae0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb953fff640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d950 tx=0x7fb94400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.383+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb944017070 con 0x7fb954102ae0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.384+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb944009c30 con 0x7fb954102ae0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.384+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9541a0720 con 0x7fb954102ae0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.384+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400f620 con 0x7fb954102ae0 2024-09-21T02:57:42.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.384+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9541a6e80 con 0x7fb954102ae0 2024-09-21T02:57:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.385+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb954103d10 con 0x7fb954102ae0 2024-09-21T02:57:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.386+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fb944005030 con 0x7fb954102ae0 2024-09-21T02:57:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.387+0000 7fb9517fa640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 0x7fb93c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.387+0000 7fb9537fe640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 0x7fb93c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:42.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.387+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fb9440e6ba0 con 0x7fb954102ae0 2024-09-21T02:57:42.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.388+0000 7fb9537fe640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 0x7fb93c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:42.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.390+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb9440ad800 con 0x7fb954102ae0 2024-09-21T02:57:42.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:42 smithi018 ceph-mon[34587]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:42.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:42 smithi123 ceph-mon[34645]: pgmap v232: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:42.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.579+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb95419b5f0 con 0x7fb954102ae0 2024-09-21T02:57:42.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.579+0000 7fb9517fa640 1 -- 172.21.15.16:0/937178722 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v70) ==== 93+0+10 (secure 0 0 0) 0x7fb944009de0 con 0x7fb954102ae0 2024-09-21T02:57:42.581 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-21T02:57:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.581+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 msgr2=0x7fb93c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 0x7fb93c0b8a70 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb948000c00 tx=0x7fb948031400 comp rx=0 tx=0).stop 2024-09-21T02:57:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 msgr2=0x7fb95419a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:42.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fb94400d950 tx=0x7fb94400de10 comp rx=0 tx=0).stop 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 shutdown_connections 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b6560 0x7fb93c0b8a70 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb954108d20 0x7fb95419fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb9541083d0 0x7fb95419aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 --2- 172.21.15.16:0/937178722 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb954102ae0 0x7fb95419a4c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 >> 172.21.15.16:0/937178722 conn(0x7fb954079760 msgr2=0x7fb9541052d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 shutdown_connections 2024-09-21T02:57:42.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:42.582+0000 7fb95a425640 1 -- 172.21.15.16:0/937178722 wait complete. 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.a on smithi016 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.d on smithi016 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.g on smithi016 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.j on smithi016 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.b on smithi018 2024-09-21T02:57:43.213 INFO:tasks.cephadm:Adding mds.e on smithi018 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.h on smithi018 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.k on smithi018 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.c on smithi123 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.f on smithi123 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.i on smithi123 2024-09-21T02:57:43.214 INFO:tasks.cephadm:Adding mds.l on smithi123 2024-09-21T02:57:43.214 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph orch apply mds all '12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l' 2024-09-21T02:57:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:43 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/937178722' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:57:43.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:43 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/937178722' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:57:43.477 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T02:57:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:43 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/937178722' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 -- 172.21.15.123:0/3926797966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1094a0 msgr2=0x7f101c1098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 --2- 172.21.15.123:0/3926797966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1094a0 0x7f101c1098b0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f1010009900 tx=0x7f101002a410 comp rx=0 tx=0).stop 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 -- 172.21.15.123:0/3926797966 shutdown_connections 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 --2- 172.21.15.123:0/3926797966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c109df0 0x7f101c110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 --2- 172.21.15.123:0/3926797966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c1094a0 0x7f101c1098b0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 --2- 172.21.15.123:0/3926797966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c103b60 0x7f101c103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 -- 172.21.15.123:0/3926797966 >> 172.21.15.123:0/3926797966 conn(0x7f101c07b850 msgr2=0x7f101c07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 -- 172.21.15.123:0/3926797966 shutdown_connections 2024-09-21T02:57:43.957 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.955+0000 7f1021ea7640 1 -- 172.21.15.123:0/3926797966 wait complete. 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 Processor -- start 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 -- start start 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 0x7f101c19a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c1094a0 0x7f101c19aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f101c113ab0 con 0x7f101c103b60 2024-09-21T02:57:43.959 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f101c113930 con 0x7f101c1094a0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.956+0000 7f1021ea7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f101c113c30 con 0x7f101c109df0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:56920/0 (socket says 172.21.15.123:56920) 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 -- 172.21.15.123:0/4209392875 learned_addr learned my addr 172.21.15.123:0/4209392875 (peer_addr_for_me v2:172.21.15.123:0/0) 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101affd640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c1094a0 0x7f101c19aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101b7fe640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 0x7f101c19a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 -- 172.21.15.123:0/4209392875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c1094a0 msgr2=0x7f101c19aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c1094a0 0x7f101c19aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 -- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 msgr2=0x7f101c19a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 0x7f101c19a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 -- 172.21.15.123:0/4209392875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f101c1a0430 con 0x7f101c109df0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101b7fe640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 0x7f101c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f101bfff640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f101400eaa0 tx=0x7f101400ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:43.960 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f101400cdb0 con 0x7f101c109df0 2024-09-21T02:57:43.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1014005020 con 0x7f101c109df0 2024-09-21T02:57:43.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f101c1a0720 con 0x7f101c109df0 2024-09-21T02:57:43.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10140116d0 con 0x7f101c109df0 2024-09-21T02:57:43.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.957+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f101c1a6e80 con 0x7f101c109df0 2024-09-21T02:57:43.961 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.958+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f101c103f70 con 0x7f101c109df0 2024-09-21T02:57:43.962 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.961+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f10140051d0 con 0x7f101c109df0 2024-09-21T02:57:43.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.962+0000 7f1018ff9640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 0x7f10040b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:43.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.962+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f10140e7560 con 0x7f101c109df0 2024-09-21T02:57:43.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.962+0000 7f101b7fe640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 0x7f10040b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:43.963 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.963+0000 7f101b7fe640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 0x7f10040b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f100c0058c0 tx=0x7f100c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:43.966 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:43.965+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1014016030 con 0x7f101c109df0 2024-09-21T02:57:44.155 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.154+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}) -- 0x7f101c063900 con 0x7f10040b6560 2024-09-21T02:57:44.163 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.162+0000 7f1018ff9640 1 -- 172.21.15.123:0/4209392875 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f101c063900 con 0x7f10040b6560 2024-09-21T02:57:44.164 INFO:teuthology.orchestra.run.smithi123.stdout:Scheduled mds.all update... 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 msgr2=0x7f10040b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 0x7f10040b8a70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f100c0058c0 tx=0x7f100c005850 comp rx=0 tx=0).stop 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 msgr2=0x7f101c19fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f101400eaa0 tx=0x7f101400ef60 comp rx=0 tx=0).stop 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 shutdown_connections 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10040b6560 0x7f10040b8a70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f101c109df0 0x7f101c19fd00 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.166 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f101c1094a0 0x7f101c19aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 --2- 172.21.15.123:0/4209392875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f101c103b60 0x7f101c19a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:44.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.164+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 >> 172.21.15.123:0/4209392875 conn(0x7f101c07b850 msgr2=0x7f101c105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:44.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.165+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 shutdown_connections 2024-09-21T02:57:44.167 INFO:teuthology.orchestra.run.smithi123.stderr:2024-09-21T02:57:44.165+0000 7f1021ea7640 1 -- 172.21.15.123:0/4209392875 wait complete. 2024-09-21T02:57:44.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:44 smithi018 ceph-mon[34587]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:44.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:44 smithi123 ceph-mon[34645]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:44.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:44 smithi016 ceph-mon[31253]: pgmap v233: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:44.804 DEBUG:teuthology.orchestra.run.smithi016:mds.a> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.a.service 2024-09-21T02:57:44.807 DEBUG:teuthology.orchestra.run.smithi016:mds.d> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.d.service 2024-09-21T02:57:44.850 DEBUG:teuthology.orchestra.run.smithi016:mds.g> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.g.service 2024-09-21T02:57:44.893 DEBUG:teuthology.orchestra.run.smithi016:mds.j> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.j.service 2024-09-21T02:57:44.936 DEBUG:teuthology.orchestra.run.smithi018:mds.b> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.b.service 2024-09-21T02:57:44.938 DEBUG:teuthology.orchestra.run.smithi018:mds.e> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.e.service 2024-09-21T02:57:44.981 DEBUG:teuthology.orchestra.run.smithi018:mds.h> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.h.service 2024-09-21T02:57:45.024 DEBUG:teuthology.orchestra.run.smithi018:mds.k> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.k.service 2024-09-21T02:57:45.067 DEBUG:teuthology.orchestra.run.smithi123:mds.c> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.c.service 2024-09-21T02:57:45.069 DEBUG:teuthology.orchestra.run.smithi123:mds.f> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.f.service 2024-09-21T02:57:45.112 DEBUG:teuthology.orchestra.run.smithi123:mds.i> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.i.service 2024-09-21T02:57:45.155 DEBUG:teuthology.orchestra.run.smithi123:mds.l> sudo journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.l.service 2024-09-21T02:57:45.197 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-09-21T02:57:45.198 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:57:45.419 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.420 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:45 smithi018 ceph-mon[34587]: Deploying daemon mds.l on smithi123 2024-09-21T02:57:45.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-09-21T02:57:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:45 smithi123 ceph-mon[34645]: Deploying daemon mds.l on smithi123 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='client.24592 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: Saving service mds.all spec with placement smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:45.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:45.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:45.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:45 smithi016 ceph-mon[31253]: Deploying daemon mds.l on smithi123 2024-09-21T02:57:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 -- 172.21.15.16:0/793160218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 msgr2=0x7ffb74104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 --2- 172.21.15.16:0/793160218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74104e40 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009860 tx=0x7ffb6402a340 comp rx=0 tx=0).stop 2024-09-21T02:57:45.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 -- 172.21.15.16:0/793160218 shutdown_connections 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 --2- 172.21.15.16:0/793160218 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb74105380 0x7ffb7410d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 --2- 172.21.15.16:0/793160218 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb7410de60 0x7ffb74110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 --2- 172.21.15.16:0/793160218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74104e40 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 -- 172.21.15.16:0/793160218 >> 172.21.15.16:0/793160218 conn(0x7ffb74079760 msgr2=0x7ffb74079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 -- 172.21.15.16:0/793160218 shutdown_connections 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.936+0000 7ffb793f9640 1 -- 172.21.15.16:0/793160218 wait complete. 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 Processor -- start 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 -- start start 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb74105380 0x7ffb74102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74113ac0 con 0x7ffb7410de60 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74113940 con 0x7ffb74105380 2024-09-21T02:57:45.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb793f9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffb74113c40 con 0x7ffb74104a30 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43484/0 (socket says 172.21.15.16:43484) 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb737fe640 1 -- 172.21.15.16:0/684118444 learned_addr learned my addr 172.21.15.16:0/684118444 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb72ffd640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.937+0000 7ffb727fc640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb74105380 0x7ffb74102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 -- 172.21.15.16:0/684118444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 msgr2=0x7ffb74102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 -- 172.21.15.16:0/684118444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb74105380 msgr2=0x7ffb74102cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb74105380 0x7ffb74102cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 -- 172.21.15.16:0/684118444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb741a8790 con 0x7ffb7410de60 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb737fe640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00eaa0 tx=0x7ffb6c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c00cdb0 con 0x7ffb7410de60 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb741a8a80 con 0x7ffb7410de60 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb6c005020 con 0x7ffb7410de60 2024-09-21T02:57:45.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb6c0116d0 con 0x7ffb7410de60 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.938+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb741a8fc0 con 0x7ffb7410de60 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.939+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb7406c1f0 con 0x7ffb7410de60 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.941+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7ffb6c0054a0 con 0x7ffb7410de60 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.942+0000 7ffb5bfff640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 0x7ffb5c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.942+0000 7ffb72ffd640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 0x7ffb5c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.942+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7ffb6c0e7550 con 0x7ffb7410de60 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.942+0000 7ffb72ffd640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 0x7ffb5c0b8a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb741041c0 tx=0x7ffb640058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:45.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:45.945+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffb6c016030 con 0x7ffb7410de60 2024-09-21T02:57:46.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.170+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffb74104e40 con 0x7ffb7410de60 2024-09-21T02:57:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.170+0000 7ffb5bfff640 1 -- 172.21.15.16:0/684118444 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+611 (secure 0 0 0) 0x7ffb74104e40 con 0x7ffb7410de60 2024-09-21T02:57:46.172 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:46.172 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":1,"btime":"2024-09-21T02:50:00:888218+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-09-21T02:57:46.172 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 1 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 msgr2=0x7ffb5c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 0x7ffb5c0b8a70 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ffb741041c0 tx=0x7ffb640058d0 comp rx=0 tx=0).stop 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 msgr2=0x7ffb741031f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c00eaa0 tx=0x7ffb6c00ef60 comp rx=0 tx=0).stop 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 shutdown_connections 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffb5c0b6560 0x7ffb5c0b8a70 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffb7410de60 0x7ffb741031f0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:46.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffb74105380 0x7ffb74102cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 --2- 172.21.15.16:0/684118444 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffb74104a30 0x7ffb74102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 >> 172.21.15.16:0/684118444 conn(0x7ffb74079760 msgr2=0x7ffb74100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 shutdown_connections 2024-09-21T02:57:46.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:46.173+0000 7ffb793f9640 1 -- 172.21.15.16:0/684118444 wait complete. 2024-09-21T02:57:46.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:46 smithi123 ceph-mon[34645]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:46.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:46 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/684118444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:46.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:46 smithi016 ceph-mon[31253]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:46.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:46 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/684118444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:46.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:46 smithi018 ceph-mon[34587]: pgmap v234: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:46.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:46 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/684118444' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:46.988 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:46 smithi123 podman[63776]: 2024-09-21 02:57:46.984512706 +0000 UTC m=+0.363406329 container init 3aead19d935a9387a36da4f79de1a96c542aa008ed6c622e3b48e8af35968a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:57:46.988 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:46 smithi123 podman[63776]: 2024-09-21 02:57:46.987327007 +0000 UTC m=+0.366220646 container start 3aead19d935a9387a36da4f79de1a96c542aa008ed6c622e3b48e8af35968a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True) 2024-09-21T02:57:47.470 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:47 smithi123 ceph-mds[63795]: -- 172.21.15.123:0/3827762552 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x563e84a05860 con 0x563e88bad600 2024-09-21T02:57:47.470 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l[63791]: starting mds.l at 2024-09-21T02:57:47.470 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:47 smithi123 bash[63776]: 3aead19d935a9387a36da4f79de1a96c542aa008ed6c622e3b48e8af35968a06 2024-09-21T02:57:47.471 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 02:57:47 smithi123 systemd[1]: Started Ceph mds.l for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:57:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.123:6832/3810750401,v1:172.21.15.123:6833/3810750401] up:boot 2024-09-21T02:57:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: fsmap 1 up:standby 2024-09-21T02:57:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:48.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:48 smithi018 ceph-mon[34587]: Deploying daemon mds.k on smithi018 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.123:6832/3810750401,v1:172.21.15.123:6833/3810750401] up:boot 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: fsmap 1 up:standby 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:48.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:48 smithi123 ceph-mon[34645]: Deploying daemon mds.k on smithi018 2024-09-21T02:57:48.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.123:6832/3810750401,v1:172.21.15.123:6833/3810750401] up:boot 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: fsmap 1 up:standby 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "l"} : dispatch 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: pgmap v235: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:48.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:48 smithi016 ceph-mon[31253]: Deploying daemon mds.k on smithi018 2024-09-21T02:57:48.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:57:49.182 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 systemd[1]: Starting Ceph mds.k for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:57:49.184 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:49.541 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 podman[62016]: 2024-09-21 02:57:49.159461299 +0000 UTC m=+0.015991811 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:57:49.541 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 podman[62016]: 2024-09-21 02:57:49.268051721 +0000 UTC m=+0.124582274 container create 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:57:49.541 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 podman[62016]: 2024-09-21 02:57:49.50383854 +0000 UTC m=+0.360369052 container init 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:57:49.541 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 podman[62016]: 2024-09-21 02:57:49.506620595 +0000 UTC m=+0.363151106 container start 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:57:49.541 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 ceph-mds[62035]: -- 172.21.15.18:0/3353496142 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560da987f860 con 0x560dada27180 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 -- 172.21.15.16:0/2274506507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f4102ac0 msgr2=0x7f88f4102ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 --2- 172.21.15.16:0/2274506507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f4102ac0 0x7f88f4102ed0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f88e4009900 tx=0x7f88e402a410 comp rx=0 tx=0).stop 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 -- 172.21.15.16:0/2274506507 shutdown_connections 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 --2- 172.21.15.16:0/2274506507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 --2- 172.21.15.16:0/2274506507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f41083b0 0x7f88f41087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 --2- 172.21.15.16:0/2274506507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f4102ac0 0x7f88f4102ed0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 -- 172.21.15.16:0/2274506507 >> 172.21.15.16:0/2274506507 conn(0x7f88f4079790 msgr2=0x7f88f4079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 -- 172.21.15.16:0/2274506507 shutdown_connections 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.748+0000 7f88fa728640 1 -- 172.21.15.16:0/2274506507 wait complete. 2024-09-21T02:57:49.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.749+0000 7f88fa728640 1 Processor -- start 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.749+0000 7f88fa728640 1 -- start start 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f4102ac0 0x7f88f41993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f419ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f88f41129c0 con 0x7f88f41083b0 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4112840 con 0x7f88f4108d00 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88fa728640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f88f4112b40 con 0x7f88f4102ac0 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43504/0 (socket says 172.21.15.16:43504) 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 -- 172.21.15.16:0/1122710188 learned_addr learned my addr 172.21.15.16:0/1122710188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88f3fff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f4102ac0 0x7f88f41993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88f8cc5640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f419ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 -- 172.21.15.16:0/1122710188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f4102ac0 msgr2=0x7f88f41993f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:49.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f4102ac0 0x7f88f41993f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 -- 172.21.15.16:0/1122710188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 msgr2=0x7f88f419ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f419ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 -- 172.21.15.16:0/1122710188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88f419f360 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88f8cc5640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.750+0000 7f88ebfff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00d740 tx=0x7f88dc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.751+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc017070 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.751+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88f419f650 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.751+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88dc00de50 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.751+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88dc00f620 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.751+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88f41a5db0 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.752+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88f4102ed0 con 0x7f88f41083b0 2024-09-21T02:57:49.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.755+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f88dc01a070 con 0x7f88f41083b0 2024-09-21T02:57:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.756+0000 7f88f1ffb640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 0x7f88e00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:49.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.756+0000 7f88f3fff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 0x7f88e00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.756+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f88dc0e6950 con 0x7f88f41083b0 2024-09-21T02:57:49.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.757+0000 7f88f3fff640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 0x7f88e00b8af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88e4031440 tx=0x7f88e4009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:49.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.759+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f88dc0ad610 con 0x7f88f41083b0 2024-09-21T02:57:49.932 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k[62031]: starting mds.k at 2024-09-21T02:57:49.932 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 bash[62016]: 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 2024-09-21T02:57:49.932 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 02:57:49 smithi018 systemd[1]: Started Ceph mds.k for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:57:49.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.998+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f88f419a520 con 0x7f88f41083b0 2024-09-21T02:57:50.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:49.999+0000 7f88f1ffb640 1 -- 172.21.15.16:0/1122710188 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1438 (secure 0 0 0) 0x7f88dc0b19f0 con 0x7f88f41083b0 2024-09-21T02:57:50.000 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:50.000 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":2,"btime":"2024-09-21T02:57:47:025971+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-09-21T02:57:50.001 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 2 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 msgr2=0x7f88e00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 0x7f88e00b8af0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f88e4031440 tx=0x7f88e4009560 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 msgr2=0x7f88f4199990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f88dc00d740 tx=0x7f88dc00dc00 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 shutdown_connections 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f88e00b65e0 0x7f88e00b8af0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f88f41083b0 0x7f88f4199990 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f88f4108d00 0x7f88f419ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 --2- 172.21.15.16:0/1122710188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88f4102ac0 0x7f88f41993f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:50.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 >> 172.21.15.16:0/1122710188 conn(0x7f88f4079790 msgr2=0x7f88f4106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 shutdown_connections 2024-09-21T02:57:50.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:50.001+0000 7f88fa728640 1 -- 172.21.15.16:0/1122710188 wait complete. 2024-09-21T02:57:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: Deploying daemon mds.j on smithi016 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1122710188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6832/3267677843,v1:172.21.15.18:6833/3267677843] up:boot 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: fsmap 2 up:standby 2024-09-21T02:57:50.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:50 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:57:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: Deploying daemon mds.j on smithi016 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1122710188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6832/3267677843,v1:172.21.15.18:6833/3267677843] up:boot 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: fsmap 2 up:standby 2024-09-21T02:57:50.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:50 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:57:51.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: pgmap v236: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: Deploying daemon mds.j on smithi016 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1122710188' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6832/3267677843,v1:172.21.15.18:6833/3267677843] up:boot 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: fsmap 2 up:standby 2024-09-21T02:57:51.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:50 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "k"} : dispatch 2024-09-21T02:57:52.111 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:52 smithi016 ceph-mon[31253]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:52.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:52 smithi018 ceph-mon[34587]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:52.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:52 smithi123 ceph-mon[34645]: pgmap v237: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:52.508 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 systemd[1]: Starting Ceph mds.j for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:57:52.765 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 podman[69846]: 2024-09-21 02:57:52.409778462 +0000 UTC m=+0.016040938 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:57:52.766 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 podman[69846]: 2024-09-21 02:57:52.625679571 +0000 UTC m=+0.231942035 container create fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, ceph=True, org.label-schema.build-date=20240731, GIT_CLEAN=True, maintainer=Guillaume Abrioux ) 2024-09-21T02:57:53.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 podman[69846]: 2024-09-21 02:57:52.861792772 +0000 UTC m=+0.468055260 container init fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T02:57:53.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 podman[69846]: 2024-09-21 02:57:52.86470338 +0000 UTC m=+0.470965857 container start fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:57:53.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 ceph-mds[69865]: -- 172.21.15.16:0/2752424300 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55efea457c20 con 0x55efee5ffa80 2024-09-21T02:57:53.267 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j[69861]: starting mds.j at 2024-09-21T02:57:53.267 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 bash[69846]: fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e 2024-09-21T02:57:53.267 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 02:57:52 smithi016 systemd[1]: Started Ceph mds.j for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:57:53.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:57:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.16:6834/3714505775,v1:172.21.15.16:6835/3714505775] up:boot 2024-09-21T02:57:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: fsmap 3 up:standby 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: Deploying daemon mds.d on smithi016 2024-09-21T02:57:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:54 smithi123 ceph-mon[34645]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:54.311 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.16:6834/3714505775,v1:172.21.15.16:6835/3714505775] up:boot 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: fsmap 3 up:standby 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: Deploying daemon mds.d on smithi016 2024-09-21T02:57:54.312 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:54 smithi016 ceph-mon[31253]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.16:6834/3714505775,v1:172.21.15.16:6835/3714505775] up:boot 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: fsmap 3 up:standby 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "j"} : dispatch 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: Deploying daemon mds.d on smithi016 2024-09-21T02:57:54.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:54 smithi018 ceph-mon[34587]: pgmap v238: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:54.748 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 -- 172.21.15.16:0/2557709476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b001083c0 msgr2=0x7f2b001087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 --2- 172.21.15.16:0/2557709476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b001083c0 0x7f2b001087d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f2ae8009900 tx=0x7f2ae802a410 comp rx=0 tx=0).stop 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 -- 172.21.15.16:0/2557709476 shutdown_connections 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 --2- 172.21.15.16:0/2557709476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b001083c0 0x7f2b001087d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 --2- 172.21.15.16:0/2557709476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00108d10 0x7f2b0010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 --2- 172.21.15.16:0/2557709476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b00102a80 0x7f2b00102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 -- 172.21.15.16:0/2557709476 >> 172.21.15.16:0/2557709476 conn(0x7f2b00079740 msgr2=0x7f2b00079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 -- 172.21.15.16:0/2557709476 shutdown_connections 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.454+0000 7f2b0673b640 1 -- 172.21.15.16:0/2557709476 wait complete. 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.455+0000 7f2b0673b640 1 Processor -- start 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.455+0000 7f2b0673b640 1 -- start start 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00102a80 0x7f2b0019b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:55.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b001083c0 0x7f2b0019b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00112980 con 0x7f2b00108d10 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00112800 con 0x7f2b001083c0 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2b00112b00 con 0x7f2b00102a80 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43430/0 (socket says 172.21.15.16:43430) 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 -- 172.21.15.16:0/3017283998 learned_addr learned my addr 172.21.15.16:0/3017283998 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2af7fff640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b001083c0 0x7f2b0019b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2afffff640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00102a80 0x7f2b0019b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 -- 172.21.15.16:0/3017283998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00102a80 msgr2=0x7f2b0019b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00102a80 0x7f2b0019b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 -- 172.21.15.16:0/3017283998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b001083c0 msgr2=0x7f2b0019b890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b001083c0 0x7f2b0019b890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 -- 172.21.15.16:0/3017283998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b001a6550 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b04cd8640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009fd0 tx=0x7f2af800def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8017070 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2af8003e20 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b001a6840 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2af8010460 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.456+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b001a6d00 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.458+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b00103d00 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.460+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2af8004020 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.461+0000 7f2afdffb640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 0x7f2aec0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.461+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2af80e6ba0 con 0x7f2b00108d10 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.462+0000 7f2afffff640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 0x7f2aec0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:55.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.462+0000 7f2afffff640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 0x7f2aec0b8aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006c6a0 tx=0x7f2af000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:55.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.465+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2af80ad800 con 0x7f2b00108d10 2024-09-21T02:57:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.695+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2b0006bf50 con 0x7f2b00108d10 2024-09-21T02:57:55.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.696+0000 7f2afdffb640 1 -- 172.21.15.16:0/3017283998 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3088 (secure 0 0 0) 0x7f2b0006bf50 con 0x7f2b00108d10 2024-09-21T02:57:55.697 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:55.697 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":4,"btime":"2024-09-21T02:57:53:013974+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T02:57:55.698 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 4 2024-09-21T02:57:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 msgr2=0x7f2aec0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 0x7f2aec0b8aa0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f2b0006c6a0 tx=0x7f2af000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 msgr2=0x7f2b0019bdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2af8009fd0 tx=0x7f2af800def0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 shutdown_connections 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2aec0b6590 0x7f2aec0b8aa0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2b00108d10 0x7f2b0019bdd0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2b001083c0 0x7f2b0019b890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 --2- 172.21.15.16:0/3017283998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2b00102a80 0x7f2b0019b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 >> 172.21.15.16:0/3017283998 conn(0x7f2b00079740 msgr2=0x7f2b001044f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 shutdown_connections 2024-09-21T02:57:55.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:55.698+0000 7f2b0673b640 1 -- 172.21.15.16:0/3017283998 wait complete. 2024-09-21T02:57:56.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:56 smithi016 ceph-mon[31253]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:56.402 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3017283998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:56.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:56 smithi123 ceph-mon[34645]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:56.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3017283998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:56.678 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 02:57:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d[70241]: starting mds.d at 2024-09-21T02:57:56.678 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 02:57:56 smithi016 bash[70212]: ce70b281ac9810d9f59cf89b6381be053325b4d958d38e71489271c81722bd03 2024-09-21T02:57:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:56 smithi018 ceph-mon[34587]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3017283998' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:57:57.015 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 02:57:56 smithi016 systemd[1]: Started Ceph mds.d for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:57:57.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: Deploying daemon mds.e on smithi018 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.16:6836/2928144675,v1:172.21.15.16:6837/2928144675] up:boot 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: fsmap 4 up:standby 2024-09-21T02:57:57.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:57 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:57:58.012 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.012 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.012 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: Deploying daemon mds.e on smithi018 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.16:6836/2928144675,v1:172.21.15.16:6837/2928144675] up:boot 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: fsmap 4 up:standby 2024-09-21T02:57:58.013 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:57 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:57:58.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: Deploying daemon mds.e on smithi018 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.16:6836/2928144675,v1:172.21.15.16:6837/2928144675] up:boot 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: fsmap 4 up:standby 2024-09-21T02:57:58.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:57 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "d"} : dispatch 2024-09-21T02:57:58.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:57:58.696 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:57:58.917 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:57:58 smithi018 ceph-mon[34587]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:57:58 smithi016 ceph-mon[31253]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:59.182 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 02:57:59 smithi018 podman[62301]: 2024-09-21 02:57:59.159217613 +0000 UTC m=+0.356729735 container start e339fbc1d0485258d85f9246a0cc949e05d8ccda0861aaa9a96f0c8b1a8f070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.174+0000 7f08074f5640 1 -- 172.21.15.16:0/2685868973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0800102a80 msgr2=0x7f0800102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.174+0000 7f08074f5640 1 --2- 172.21.15.16:0/2685868973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0800102a80 0x7f0800102e90 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f07fc009900 tx=0x7f07fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 -- 172.21.15.16:0/2685868973 shutdown_connections 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 --2- 172.21.15.16:0/2685868973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f080010f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 --2- 172.21.15.16:0/2685868973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f08001083c0 0x7f08001087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 --2- 172.21.15.16:0/2685868973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0800102a80 0x7f0800102e90 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 -- 172.21.15.16:0/2685868973 >> 172.21.15.16:0/2685868973 conn(0x7f0800079740 msgr2=0x7f0800079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 -- 172.21.15.16:0/2685868973 shutdown_connections 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 -- 172.21.15.16:0/2685868973 wait complete. 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 Processor -- start 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.175+0000 7f08074f5640 1 -- start start 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0800102a80 0x7f08001990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:59.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f0800199b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f08001129e0 con 0x7f08001083c0 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112860 con 0x7f0800102a80 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f08074f5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0800112b60 con 0x7f0800108d10 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43456/0 (socket says 172.21.15.16:43456) 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 -- 172.21.15.16:0/2291149147 learned_addr learned my addr 172.21.15.16:0/2291149147 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0805291640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0800102a80 0x7f08001990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0805a92640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f0800199b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 -- 172.21.15.16:0/2291149147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 msgr2=0x7f0800199b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f0800199b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 -- 172.21.15.16:0/2291149147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0800102a80 msgr2=0x7f08001990e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0800102a80 0x7f08001990e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 -- 172.21.15.16:0/2291149147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f08001a8790 con 0x7f08001083c0 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0805a92640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f0800199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:57:59.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.176+0000 7f0804a90640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e9c0 tx=0x7f07f400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.177+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f400cd80 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.177+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f08001a8a80 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.177+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f40054d0 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.177+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4010430 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.177+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08001a9040 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.178+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0800103d00 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.180+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f07f4004fb0 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.181+0000 7f07f27fc640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.181+0000 7f0805291640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.181+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f07f40e76f0 con 0x7f08001083c0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.182+0000 7f0805291640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 0x7f07ec0b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:57:59.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.185+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f07f4056710 con 0x7f08001083c0 2024-09-21T02:57:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:57:58 smithi123 ceph-mon[34645]: pgmap v240: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:57:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.411+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f080006bf50 con 0x7f08001083c0 2024-09-21T02:57:59.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.412+0000 7f07f27fc640 1 -- 172.21.15.16:0/2291149147 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3913 (secure 0 0 0) 0x7f080006bf50 con 0x7f08001083c0 2024-09-21T02:57:59.414 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:57:59.414 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":5,"btime":"2024-09-21T02:57:57:187761+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-09-21T02:57:59.414 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 5 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 msgr2=0x7f07ec0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 0x7f07ec0b8a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f07fc000c00 tx=0x7f07fc031400 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 msgr2=0x7f0800199620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f07f400e9c0 tx=0x7f07f400ee80 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 shutdown_connections 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f07ec0b6560 0x7f07ec0b8a70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f08001083c0 0x7f0800199620 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0800108d10 0x7f0800199b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 --2- 172.21.15.16:0/2291149147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0800102a80 0x7f08001990e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:57:59.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 >> 172.21.15.16:0/2291149147 conn(0x7f0800079740 msgr2=0x7f0800101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:57:59.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 shutdown_connections 2024-09-21T02:57:59.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:57:59.415+0000 7f08074f5640 1 -- 172.21.15.16:0/2291149147 wait complete. 2024-09-21T02:57:59.682 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 02:57:59 smithi018 ceph-mds[62320]: -- 172.21.15.18:0/2668675795 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b19fe4f860 con 0x55b1a3ff7180 2024-09-21T02:57:59.682 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 02:57:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e[62316]: starting mds.e at 2024-09-21T02:57:59.682 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 02:57:59 smithi018 bash[62301]: e339fbc1d0485258d85f9246a0cc949e05d8ccda0861aaa9a96f0c8b1a8f070e 2024-09-21T02:57:59.682 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 02:57:59 smithi018 systemd[1]: Started Ceph mds.e for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:00.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: Deploying daemon mds.f on smithi123 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2291149147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6834/3014746634,v1:172.21.15.18:6835/3014746634] up:boot 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: fsmap 5 up:standby 2024-09-21T02:58:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:00 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:58:00.656 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:00.656 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.656 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.656 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: Deploying daemon mds.f on smithi123 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2291149147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6834/3014746634,v1:172.21.15.18:6835/3014746634] up:boot 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: fsmap 5 up:standby 2024-09-21T02:58:00.657 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:00 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: Deploying daemon mds.f on smithi123 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2291149147' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6834/3014746634,v1:172.21.15.18:6835/3014746634] up:boot 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: fsmap 5 up:standby 2024-09-21T02:58:00.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:00 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "e"} : dispatch 2024-09-21T02:58:01.358 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 systemd[1]: Starting Ceph mds.f for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:01.639 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 podman[64043]: 2024-09-21 02:58:01.260172516 +0000 UTC m=+0.015643409 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:01.639 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 podman[64043]: 2024-09-21 02:58:01.38269505 +0000 UTC m=+0.138165942 container create 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD) 2024-09-21T02:58:01.639 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 podman[64043]: 2024-09-21 02:58:01.602012091 +0000 UTC m=+0.357482984 container init 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-09-21T02:58:01.639 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 podman[64043]: 2024-09-21 02:58:01.604778334 +0000 UTC m=+0.360249236 container start 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-09-21T02:58:01.970 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 ceph-mds[64062]: -- 172.21.15.123:0/2099569485 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e4e271860 con 0x558e52419600 2024-09-21T02:58:01.970 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f[64058]: starting mds.f at 2024-09-21T02:58:01.970 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 bash[64043]: 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 2024-09-21T02:58:01.970 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 02:58:01 smithi123 systemd[1]: Started Ceph mds.f for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:02.210 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:02.480 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: Deploying daemon mds.c on smithi123 2024-09-21T02:58:02.915 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.123:6834/4189028695,v1:172.21.15.123:6835/4189028695] up:boot 2024-09-21T02:58:02.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: fsmap 6 up:standby 2024-09-21T02:58:02.916 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:02 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 -- 172.21.15.16:0/1751506385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 msgr2=0x7f52c81087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1751506385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81087d0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f52b8009900 tx=0x7f52b802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 -- 172.21.15.16:0/1751506385 shutdown_connections 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1751506385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81087d0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1751506385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8108d10 0x7f52c810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1751506385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8102a80 0x7f52c8102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 -- 172.21.15.16:0/1751506385 >> 172.21.15.16:0/1751506385 conn(0x7f52c8079740 msgr2=0x7f52c8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:02.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 -- 172.21.15.16:0/1751506385 shutdown_connections 2024-09-21T02:58:02.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.938+0000 7f52cfb08640 1 -- 172.21.15.16:0/1751506385 wait complete. 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.939+0000 7f52cfb08640 1 Processor -- start 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.939+0000 7f52cfb08640 1 -- start start 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.939+0000 7f52cfb08640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 0x7f52c8199410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cfb08640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cfb08640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8108d10 0x7f52c819ec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:02.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cfb08640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8112980 con 0x7f52c81083c0 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cfb08640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8112800 con 0x7f52c8108d10 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cfb08640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f52c8112b00 con 0x7f52c8102a80 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd8a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 0x7f52c8199410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43474/0 (socket says 172.21.15.16:43474) 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 -- 172.21.15.16:0/1784453655 learned_addr learned my addr 172.21.15.16:0/1784453655 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd8a4640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 0x7f52c8199410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:43106/0 (socket says 172.21.15.16:43106) 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52ce0a5640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8108d10 0x7f52c819ec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 -- 172.21.15.16:0/1784453655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 msgr2=0x7f52c8199410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 0x7f52c8199410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 -- 172.21.15.16:0/1784453655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8108d10 msgr2=0x7f52c819ec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8108d10 0x7f52c819ec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 -- 172.21.15.16:0/1784453655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52c819f380 con 0x7f52c81083c0 2024-09-21T02:58:02.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52cd0a3640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52b8032fd0 tx=0x7f52b8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b803f030 con 0x7f52c81083c0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.940+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52b8004490 con 0x7f52c81083c0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.941+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52c819f610 con 0x7f52c81083c0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.941+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52b8035070 con 0x7f52c81083c0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.941+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52c81a5dd0 con 0x7f52c81083c0 2024-09-21T02:58:02.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.942+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52c8103d00 con 0x7f52c81083c0 2024-09-21T02:58:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.943+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f52b8042030 con 0x7f52c81083c0 2024-09-21T02:58:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.944+0000 7f52beffd640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 0x7f52b40b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:02.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.944+0000 7f52cd8a4640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 0x7f52b40b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.944+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f52b8107880 con 0x7f52c81083c0 2024-09-21T02:58:02.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.945+0000 7f52cd8a4640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 0x7f52b40b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52c400bbe0 tx=0x7f52c400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:02.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:02.947+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f52b80d2820 con 0x7f52c81083c0 2024-09-21T02:58:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.172+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f52c8102e90 con 0x7f52c81083c0 2024-09-21T02:58:03.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.173+0000 7f52beffd640 1 -- 172.21.15.16:0/1784453655 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5566 (secure 0 0 0) 0x7f52b80d7760 con 0x7f52c81083c0 2024-09-21T02:58:03.174 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:03.175 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":7,"btime":"2024-09-21T02:58:02:012290+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-09-21T02:58:03.175 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 7 2024-09-21T02:58:03.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 msgr2=0x7f52b40b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 0x7f52b40b8a70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f52c400bbe0 tx=0x7f52c400b390 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 msgr2=0x7f52c81999b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f52b8032fd0 tx=0x7f52b8004270 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 shutdown_connections 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f52b40b6560 0x7f52b40b8a70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f52c81083c0 0x7f52c81999b0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f52c8108d10 0x7f52c819ec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 --2- 172.21.15.16:0/1784453655 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f52c8102a80 0x7f52c8199410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 >> 172.21.15.16:0/1784453655 conn(0x7f52c8079740 msgr2=0x7f52c8104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 shutdown_connections 2024-09-21T02:58:03.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:03.176+0000 7f52cfb08640 1 -- 172.21.15.16:0/1784453655 wait complete. 2024-09-21T02:58:03.181 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:03.181 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.181 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: Deploying daemon mds.c on smithi123 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.123:6834/4189028695,v1:172.21.15.123:6835/4189028695] up:boot 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: fsmap 6 up:standby 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:02 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: Deploying daemon mds.c on smithi123 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.123:6834/4189028695,v1:172.21.15.123:6835/4189028695] up:boot 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: fsmap 6 up:standby 2024-09-21T02:58:03.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:02 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "f"} : dispatch 2024-09-21T02:58:03.770 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:03 smithi123 systemd[1]: Starting Ceph mds.c for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:04.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:03 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1784453655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:04.034 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:03 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1784453655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:04.035 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:03 smithi123 podman[64310]: 2024-09-21 02:58:03.809074259 +0000 UTC m=+0.015824906 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:04.035 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:03 smithi123 podman[64310]: 2024-09-21 02:58:03.910495982 +0000 UTC m=+0.117246671 container create 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True) 2024-09-21T02:58:04.084 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:03 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1784453655' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 podman[64310]: 2024-09-21 02:58:04.154434199 +0000 UTC m=+0.361184847 container init 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 podman[64310]: 2024-09-21 02:58:04.157190207 +0000 UTC m=+0.363940855 container start 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mds[64329]: -- 172.21.15.123:0/2440843204 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55781b441860 con 0x55781f5e9600 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c[64325]: starting mds.c at 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 bash[64310]: 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 2024-09-21T02:58:04.471 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 02:58:04 smithi123 systemd[1]: Started Ceph mds.c for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:04.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:04 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:05.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:04 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:05.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:05.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:04 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:06.065 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:05 smithi018 ceph-mon[34587]: Deploying daemon mds.h on smithi018 2024-09-21T02:58:06.065 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:05 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.123:6836/2688429163,v1:172.21.15.123:6837/2688429163] up:boot 2024-09-21T02:58:06.065 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:05 smithi018 ceph-mon[34587]: fsmap 7 up:standby 2024-09-21T02:58:06.065 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:05 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:58:06.070 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:06.099 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:05 smithi016 ceph-mon[31253]: Deploying daemon mds.h on smithi018 2024-09-21T02:58:06.099 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:05 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.123:6836/2688429163,v1:172.21.15.123:6837/2688429163] up:boot 2024-09-21T02:58:06.099 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:05 smithi016 ceph-mon[31253]: fsmap 7 up:standby 2024-09-21T02:58:06.099 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:05 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:58:06.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:05 smithi123 ceph-mon[34645]: Deploying daemon mds.h on smithi018 2024-09-21T02:58:06.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:05 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.123:6836/2688429163,v1:172.21.15.123:6837/2688429163] up:boot 2024-09-21T02:58:06.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:05 smithi123 ceph-mon[34645]: fsmap 7 up:standby 2024-09-21T02:58:06.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:05 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "c"} : dispatch 2024-09-21T02:58:06.341 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:06.433 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 systemd[1]: Starting Ceph mds.h for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:06.737 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 podman[62583]: 2024-09-21 02:58:06.359333902 +0000 UTC m=+0.016494135 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:06.738 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 podman[62583]: 2024-09-21 02:58:06.488500828 +0000 UTC m=+0.145661050 container create 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:58:06.738 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 podman[62583]: 2024-09-21 02:58:06.701797505 +0000 UTC m=+0.358957743 container init 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T02:58:06.738 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 podman[62583]: 2024-09-21 02:58:06.704614715 +0000 UTC m=+0.361774941 container start 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 -- 172.21.15.16:0/963341286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298103b70 msgr2=0x7f1298103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 --2- 172.21.15.16:0/963341286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298103b70 0x7f1298103f80 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f1294009900 tx=0x7f129402a410 comp rx=0 tx=0).stop 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 -- 172.21.15.16:0/963341286 shutdown_connections 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 --2- 172.21.15.16:0/963341286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298109e00 0x7f1298110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 --2- 172.21.15.16:0/963341286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f12981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 --2- 172.21.15.16:0/963341286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298103b70 0x7f1298103f80 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:06.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 -- 172.21.15.16:0/963341286 >> 172.21.15.16:0/963341286 conn(0x7f12980ff5e0 msgr2=0x7f1298101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 -- 172.21.15.16:0/963341286 shutdown_connections 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.800+0000 7f12a030e640 1 -- 172.21.15.16:0/963341286 wait complete. 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 Processor -- start 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 -- start start 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f129819ab50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1298113b00 con 0x7f1298109e00 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1298113980 con 0x7f1298103b70 2024-09-21T02:58:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f12a030e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1298113c80 con 0x7f12981094b0 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f129e8ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f129e0aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f129e8ab640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54376/0 (socket says 172.21.15.16:54376) 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f129e0aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:33458/0 (socket says 172.21.15.16:33458) 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.801+0000 7f129e8ab640 1 -- 172.21.15.16:0/1031112370 learned_addr learned my addr 172.21.15.16:0/1031112370 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129d8a9640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f129819ab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 -- 172.21.15.16:0/1031112370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 msgr2=0x7f129819ab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f129819ab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 -- 172.21.15.16:0/1031112370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 msgr2=0x7f129819a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 -- 172.21.15.16:0/1031112370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12981a0460 con 0x7f1298109e00 2024-09-21T02:58:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129d8a9640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f129819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e0aa640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f129e8ab640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f129000cce0 tx=0x7f129000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f129000f450 con 0x7f1298109e00 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f129000faa0 con 0x7f1298109e00 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12981a0750 con 0x7f1298109e00 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1290018b70 con 0x7f1298109e00 2024-09-21T02:58:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.802+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12981a6f70 con 0x7f1298109e00 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.803+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1298104df0 con 0x7f1298109e00 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.805+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f1290018d10 con 0x7f1298109e00 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.806+0000 7f128f7fe640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 0x7f12840b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.806+0000 7f129e0aa640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 0x7f12840b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.806+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f12900e7510 con 0x7f1298109e00 2024-09-21T02:58:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.806+0000 7f129e0aa640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 0x7f12840b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1294031440 tx=0x7f1294032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:06.809+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f1290014030 con 0x7f1298109e00 2024-09-21T02:58:07.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.043+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1298103f80 con 0x7f1298109e00 2024-09-21T02:58:07.045 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:06 smithi016 ceph-mon[31253]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.044+0000 7f128f7fe640 1 -- 172.21.15.16:0/1031112370 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7219 (secure 0 0 0) 0x7f1298103f80 con 0x7f1298109e00 2024-09-21T02:58:07.046 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:07.046 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":9,"btime":"2024-09-21T02:58:06:780592+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:58:07.046 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 9 2024-09-21T02:58:07.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 msgr2=0x7f12840b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 0x7f12840b8aa0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f1294031440 tx=0x7f1294032ed0 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 msgr2=0x7f129819fdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f129000cce0 tx=0x7f129000a620 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 shutdown_connections 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f12840b6590 0x7f12840b8aa0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1298109e00 0x7f129819fdf0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12981094b0 0x7f129819ab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 --2- 172.21.15.16:0/1031112370 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1298103b70 0x7f129819a5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 >> 172.21.15.16:0/1031112370 conn(0x7f12980ff5e0 msgr2=0x7f1298101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 shutdown_connections 2024-09-21T02:58:07.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:07.047+0000 7f12a030e640 1 -- 172.21.15.16:0/1031112370 wait complete. 2024-09-21T02:58:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:06 smithi018 ceph-mon[34587]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:07.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 ceph-mds[62603]: -- 172.21.15.18:0/3707270637 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55849a939860 con 0x55849eae1600 2024-09-21T02:58:07.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: starting mds.h at 2024-09-21T02:58:07.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 bash[62583]: 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 2024-09-21T02:58:07.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 02:58:06 smithi018 systemd[1]: Started Ceph mds.h for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:06 smithi123 ceph-mon[34645]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:07.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6836/2180643175,v1:172.21.15.18:6837/2180643175] up:boot 2024-09-21T02:58:07.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: fsmap 8 up:standby 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: Deploying daemon mds.i on smithi123 2024-09-21T02:58:07.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:07 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1031112370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6836/2180643175,v1:172.21.15.18:6837/2180643175] up:boot 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: fsmap 8 up:standby 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:08.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:08.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: Deploying daemon mds.i on smithi123 2024-09-21T02:58:08.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:07 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1031112370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:08.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6836/2180643175,v1:172.21.15.18:6837/2180643175] up:boot 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: fsmap 8 up:standby 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: Deploying daemon mds.i on smithi123 2024-09-21T02:58:08.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:07 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1031112370' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:08.868 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:08 smithi123 systemd[1]: Starting Ceph mds.i for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:09.127 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:08 smithi123 ceph-mon[34645]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:09.127 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:08 smithi123 podman[64580]: 2024-09-21 02:58:08.849579918 +0000 UTC m=+0.017205658 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:09.127 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:08 smithi123 podman[64580]: 2024-09-21 02:58:08.996376161 +0000 UTC m=+0.164001899 container create 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:58:09.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:08 smithi018 ceph-mon[34587]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:09.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:08 smithi016 ceph-mon[31253]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:09.693 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:09.720 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 podman[64580]: 2024-09-21 02:58:09.240362745 +0000 UTC m=+0.407988482 container init 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T02:58:09.721 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 podman[64580]: 2024-09-21 02:58:09.243128772 +0000 UTC m=+0.410754509 container start 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-09-21T02:58:09.721 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 ceph-mds[64599]: -- 172.21.15.123:0/1040389026 <== mon.1 v2:172.21.15.123:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x555b98e0d860 con 0x555b9cfb5600 2024-09-21T02:58:09.721 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i[64595]: starting mds.i at 2024-09-21T02:58:09.721 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 bash[64580]: 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 2024-09-21T02:58:09.721 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 02:58:09 smithi123 systemd[1]: Started Ceph mds.i for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:10.631 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: Deploying daemon mds.g on smithi016 2024-09-21T02:58:10.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.123:6838/1332368204,v1:172.21.15.123:6839/1332368204] up:boot 2024-09-21T02:58:10.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: fsmap 9 up:standby 2024-09-21T02:58:10.632 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:10 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: Deploying daemon mds.g on smithi016 2024-09-21T02:58:10.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.123:6838/1332368204,v1:172.21.15.123:6839/1332368204] up:boot 2024-09-21T02:58:10.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: fsmap 9 up:standby 2024-09-21T02:58:10.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:10 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:58:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: Deploying daemon mds.g on smithi016 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.123:6838/1332368204,v1:172.21.15.123:6839/1332368204] up:boot 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: fsmap 9 up:standby 2024-09-21T02:58:10.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:10 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "i"} : dispatch 2024-09-21T02:58:10.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:11.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 -- 172.21.15.16:0/3323449183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df41049c0 msgr2=0x7f7df4104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 --2- 172.21.15.16:0/3323449183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df41049c0 0x7f7df4104dd0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f7de4009900 tx=0x7f7de402a410 comp rx=0 tx=0).stop 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 -- 172.21.15.16:0/3323449183 shutdown_connections 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 --2- 172.21.15.16:0/3323449183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df410de20 0x7f7df4110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 --2- 172.21.15.16:0/3323449183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df4105310 0x7f7df410d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 --2- 172.21.15.16:0/3323449183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df41049c0 0x7f7df4104dd0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 -- 172.21.15.16:0/3323449183 >> 172.21.15.16:0/3323449183 conn(0x7f7df4079730 msgr2=0x7f7df4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 -- 172.21.15.16:0/3323449183 shutdown_connections 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.625+0000 7f7df9d61640 1 -- 172.21.15.16:0/3323449183 wait complete. 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 Processor -- start 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 -- start start 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 0x7f7df419a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4105310 0x7f7df419aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:11.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4113aa0 con 0x7f7df410de20 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4113920 con 0x7f7df41049c0 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.626+0000 7f7df9d61640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7df4113c20 con 0x7f7df4105310 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:54390/0 (socket says 172.21.15.16:54390) 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 0x7f7df419a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 -- 172.21.15.16:0/1102591081 learned_addr learned my addr 172.21.15.16:0/1102591081 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df2ffd640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4105310 0x7f7df419aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 -- 172.21.15.16:0/1102591081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4105310 msgr2=0x7f7df419aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4105310 0x7f7df419aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 -- 172.21.15.16:0/1102591081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 msgr2=0x7f7df419a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 0x7f7df419a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 -- 172.21.15.16:0/1102591081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7df41a0400 con 0x7f7df410de20 2024-09-21T02:58:11.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df37fe640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 0x7f7df419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df3fff640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec004b20 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7df41a06f0 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7dec004cc0 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7dec010670 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.627+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7df41a6e50 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.628+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7df406c1e0 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.631+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f7dec005000 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.632+0000 7f7df0ff9640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 0x7f7ddc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.632+0000 7f7df37fe640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 0x7f7ddc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.632+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f7dec0e7700 con 0x7f7df410de20 2024-09-21T02:58:11.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.632+0000 7f7df37fe640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 0x7f7ddc0b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7de4000c00 tx=0x7f7de4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:11.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.635+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7dec056710 con 0x7f7df410de20 2024-09-21T02:58:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.880+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7df4104dd0 con 0x7f7df410de20 2024-09-21T02:58:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.881+0000 7f7df0ff9640 1 -- 172.21.15.16:0/1102591081 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8049 (secure 0 0 0) 0x7f7df4104dd0 con 0x7f7df410de20 2024-09-21T02:58:11.883 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:11.883 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":10,"btime":"2024-09-21T02:58:09:866856+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:58:11.883 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 10 2024-09-21T02:58:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 msgr2=0x7f7ddc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 0x7f7ddc0b8a70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f7de4000c00 tx=0x7f7de4031400 comp rx=0 tx=0).stop 2024-09-21T02:58:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 msgr2=0x7f7df419fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f7dec00eb10 tx=0x7f7dec00c2d0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 shutdown_connections 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7ddc0b6560 0x7f7ddc0b8a70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7df410de20 0x7f7df419fcd0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7df4105310 0x7f7df419aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 --2- 172.21.15.16:0/1102591081 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7df41049c0 0x7f7df419a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 >> 172.21.15.16:0/1102591081 conn(0x7f7df4079730 msgr2=0x7f7df41007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 shutdown_connections 2024-09-21T02:58:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:11.884+0000 7f7df9d61640 1 -- 172.21.15.16:0/1102591081 wait complete. 2024-09-21T02:58:12.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:12 smithi016 ceph-mon[31253]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:12.590 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:12 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1102591081' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:12.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:12 smithi018 ceph-mon[34587]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:12.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:12 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1102591081' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:12.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:12 smithi123 ceph-mon[34645]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:12.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:12 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1102591081' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:13.015 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 02:58:12 smithi016 systemd[1]: Started Ceph mds.g for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:14.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: Deploying daemon mds.a on smithi016 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.16:6838/2173086680,v1:172.21.15.16:6839/2173086680] up:boot 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: fsmap 10 up:standby 2024-09-21T02:58:14.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:13 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:58:14.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.206 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: Deploying daemon mds.a on smithi016 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.16:6838/2173086680,v1:172.21.15.16:6839/2173086680] up:boot 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: fsmap 10 up:standby 2024-09-21T02:58:14.207 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:13 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:58:14.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: Deploying daemon mds.a on smithi016 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.16:6838/2173086680,v1:172.21.15.16:6839/2173086680] up:boot 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: fsmap 10 up:standby 2024-09-21T02:58:14.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:13 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "g"} : dispatch 2024-09-21T02:58:14.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:14.765 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:14 smithi016 systemd[1]: Starting Ceph mds.a for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:15.094 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:14 smithi018 ceph-mon[34587]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:15.094 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:14 smithi016 ceph-mon[31253]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:15.095 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:14 smithi016 podman[71254]: 2024-09-21 02:58:14.821343763 +0000 UTC m=+0.019682679 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:15.095 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:14 smithi016 podman[71254]: 2024-09-21 02:58:14.966628905 +0000 UTC m=+0.164967843 container create 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, org.label-schema.license=GPLv2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T02:58:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:14 smithi123 ceph-mon[34645]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:15.347 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:15.472 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 podman[71254]: 2024-09-21 02:58:15.194456252 +0000 UTC m=+0.392795149 container init 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T02:58:15.472 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 podman[71254]: 2024-09-21 02:58:15.197970741 +0000 UTC m=+0.396309647 container start 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux ) 2024-09-21T02:58:15.473 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 ceph-mds[71296]: -- 172.21.15.16:0/2208763500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55ec21013860 con 0x55ec251bba80 2024-09-21T02:58:15.473 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a[71280]: starting mds.a at 2024-09-21T02:58:15.473 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 bash[71254]: 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 2024-09-21T02:58:15.473 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 02:58:15 smithi016 systemd[1]: Started Ceph mds.a for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 -- 172.21.15.16:0/4292800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 msgr2=0x7f2df010f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 --2- 172.21.15.16:0/4292800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df010f530 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2de8009ca0 tx=0x7f2de802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 -- 172.21.15.16:0/4292800687 shutdown_connections 2024-09-21T02:58:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 --2- 172.21.15.16:0/4292800687 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df010f530 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 --2- 172.21.15.16:0/4292800687 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df01087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 --2- 172.21.15.16:0/4292800687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df0102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 -- 172.21.15.16:0/4292800687 >> 172.21.15.16:0/4292800687 conn(0x7f2df0079710 msgr2=0x7f2df0079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 -- 172.21.15.16:0/4292800687 shutdown_connections 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.824+0000 7f2df7143640 1 -- 172.21.15.16:0/4292800687 wait complete. 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 Processor -- start 2024-09-21T02:58:15.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 -- start start 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df019b230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df019b770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2df01129c0 con 0x7f2df0108d00 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0112840 con 0x7f2df01083b0 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df7143640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2df0112b40 con 0x7f2df0102a70 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df56e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df56e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51754/0 (socket says 172.21.15.16:51754) 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2df56e0640 1 -- 172.21.15.16:0/327343338 learned_addr learned my addr 172.21.15.16:0/327343338 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:15.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.825+0000 7f2deffff640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df019b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df4edf640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df019b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 -- 172.21.15.16:0/327343338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 msgr2=0x7f2df019b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df019b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 -- 172.21.15.16:0/327343338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 msgr2=0x7f2df019b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df019b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 -- 172.21.15.16:0/327343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2df01a6540 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df4edf640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2deffff640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df019b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df56e0640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005cc0 tx=0x7f2de802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de8003bf0 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2de8003d90 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2df01a67d0 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2de802fbc0 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.826+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2df01a6d60 con 0x7f2df0108d00 2024-09-21T02:58:15.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.827+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2df0103cf0 con 0x7f2df0108d00 2024-09-21T02:58:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.829+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2de802fd60 con 0x7f2df0108d00 2024-09-21T02:58:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.830+0000 7f2dedffb640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 0x7f2de00b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.830+0000 7f2df4edf640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 0x7f2de00b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:15.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.830+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f2de8108660 con 0x7f2df0108d00 2024-09-21T02:58:15.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.831+0000 7f2df4edf640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 0x7f2de00b8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005fd0 tx=0x7f2de4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:15.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:15.833+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2de80cf220 con 0x7f2df0108d00 2024-09-21T02:58:16.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.065+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f2df0102e80 con 0x7f2df0108d00 2024-09-21T02:58:16.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.065+0000 7f2dedffb640 1 -- 172.21.15.16:0/327343338 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9701 (secure 0 0 0) 0x7f2de80d3600 con 0x7f2df0108d00 2024-09-21T02:58:16.067 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:16.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":12,"btime":"2024-09-21T02:58:15:841549+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-09-21T02:58:16.067 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 12 2024-09-21T02:58:16.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 msgr2=0x7f2de00b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 0x7f2de00b8af0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f2de4005fd0 tx=0x7f2de4005850 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 msgr2=0x7f2df019bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f2de8005cc0 tx=0x7f2de802f380 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 shutdown_connections 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2de00b65e0 0x7f2de00b8af0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2df0108d00 0x7f2df019bcb0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2df01083b0 0x7f2df019b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 --2- 172.21.15.16:0/327343338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2df0102a70 0x7f2df019b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 >> 172.21.15.16:0/327343338 conn(0x7f2df0079710 msgr2=0x7f2df01063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 shutdown_connections 2024-09-21T02:58:16.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:16.068+0000 7f2df7143640 1 -- 172.21.15.16:0/327343338 wait complete. 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:16.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: Deploying daemon mds.b on smithi018 2024-09-21T02:58:16.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.16:6840/3872585849,v1:172.21.15.16:6841/3872585849] up:boot 2024-09-21T02:58:16.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: fsmap 11 up:standby 2024-09-21T02:58:16.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:58:16.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/327343338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:16.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: Deploying daemon mds.b on smithi018 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.16:6840/3872585849,v1:172.21.15.16:6841/3872585849] up:boot 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: fsmap 11 up:standby 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:58:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/327343338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:16.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]} : dispatch 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: Deploying daemon mds.b on smithi018 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.16:6840/3872585849,v1:172.21.15.16:6841/3872585849] up:boot 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: fsmap 11 up:standby 2024-09-21T02:58:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "a"} : dispatch 2024-09-21T02:58:16.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/327343338' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:17.432 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 systemd[1]: Starting Ceph mds.b for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T02:58:17.782 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 podman[62851]: 2024-09-21 02:58:17.414731544 +0000 UTC m=+0.016392334 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T02:58:17.782 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 podman[62851]: 2024-09-21 02:58:17.535055358 +0000 UTC m=+0.136716189 container create 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T02:58:17.782 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 podman[62851]: 2024-09-21 02:58:17.745653332 +0000 UTC m=+0.347314122 container init 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T02:58:17.782 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 podman[62851]: 2024-09-21 02:58:17.74846582 +0000 UTC m=+0.350126609 container start 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T02:58:18.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 ceph-mds[62870]: -- 172.21.15.18:0/2632839271 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cd0a089860 con 0x55cd0e231a80 2024-09-21T02:58:18.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: starting mds.b at 2024-09-21T02:58:18.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 bash[62851]: 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 2024-09-21T02:58:18.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:17 smithi018 systemd[1]: Started Ceph mds.b for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T02:58:18.850 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:19.117 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:boot 2024-09-21T02:58:19.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: fsmap 12 up:standby 2024-09-21T02:58:19.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:58:19.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:18 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:boot 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: fsmap 12 up:standby 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:58:19.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:18 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: pgmap v250: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.240 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:19.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:boot 2024-09-21T02:58:19.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: fsmap 12 up:standby 2024-09-21T02:58:19.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T02:58:19.241 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:18 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 -- 172.21.15.16:0/1019456031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c110190 msgr2=0x7fd15c1125b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 --2- 172.21.15.16:0/1019456031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c110190 0x7fd15c1125b0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fd154009ca0 tx=0x7fd15402a950 comp rx=0 tx=0).stop 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 -- 172.21.15.16:0/1019456031 shutdown_connections 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 --2- 172.21.15.16:0/1019456031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c110190 0x7fd15c1125b0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 --2- 172.21.15.16:0/1019456031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c102f30 0x7fd15c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 --2- 172.21.15.16:0/1019456031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c1025e0 0x7fd15c1029f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 -- 172.21.15.16:0/1019456031 >> 172.21.15.16:0/1019456031 conn(0x7fd15c079710 msgr2=0x7fd15c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 -- 172.21.15.16:0/1019456031 shutdown_connections 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.624+0000 7fd163f68640 1 -- 172.21.15.16:0/1019456031 wait complete. 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 Processor -- start 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 -- start start 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 0x7fd15c19cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c110190 0x7fd15c1a1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c115cc0 con 0x7fd15c1025e0 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c115b40 con 0x7fd15c110190 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd163f68640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd15c115e40 con 0x7fd15c102f30 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51764/0 (socket says 172.21.15.16:51764) 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 -- 172.21.15.16:0/1223365779 learned_addr learned my addr 172.21.15.16:0/1223365779 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161503640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 0x7fd15c19cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd162505640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c110190 0x7fd15c1a1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 -- 172.21.15.16:0/1223365779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 msgr2=0x7fd15c19cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 0x7fd15c19cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 -- 172.21.15.16:0/1223365779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c110190 msgr2=0x7fd15c1a1e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.625+0000 7fd161d04640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c110190 0x7fd15c1a1e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd161d04640 1 -- 172.21.15.16:0/1223365779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd15c1a25c0 con 0x7fd15c1025e0 2024-09-21T02:58:19.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd161503640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 0x7fd15c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd161d04640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd15800b4a0 tx=0x7fd15800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158014070 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd158010ac0 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd15c1a28b0 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd158017c30 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.626+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd15c1a9010 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.627+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd15c06c280 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.628+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fd158017070 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.629+0000 7fd152ffd640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 0x7fd1480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.629+0000 7fd161503640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 0x7fd1480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.629+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fd1580e65b0 con 0x7fd15c1025e0 2024-09-21T02:58:19.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.630+0000 7fd161503640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 0x7fd1480b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005d00 tx=0x7fd14c005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:19.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.634+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd1580ad210 con 0x7fd15c1025e0 2024-09-21T02:58:19.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.876+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd15c19d780 con 0x7fd15c1025e0 2024-09-21T02:58:19.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.877+0000 7fd152ffd640 1 -- 172.21.15.16:0/1223365779 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10524 (secure 0 0 0) 0x7fd15c19d780 con 0x7fd15c1025e0 2024-09-21T02:58:19.880 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:19.880 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":13,"btime":"2024-09-21T02:58:18:346854+0000","default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24674,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24701,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-09-21T02:58:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 13 2024-09-21T02:58:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.879+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 msgr2=0x7fd1480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.879+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 0x7fd1480b8a70 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd14c005d00 tx=0x7fd14c005c70 comp rx=0 tx=0).stop 2024-09-21T02:58:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.879+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 msgr2=0x7fd15c19c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.879+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7fd15800b4a0 tx=0x7fd15800b960 comp rx=0 tx=0).stop 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 shutdown_connections 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd1480b6560 0x7fd1480b8a70 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd15c110190 0x7fd15c1a1e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd15c102f30 0x7fd15c19cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 --2- 172.21.15.16:0/1223365779 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd15c1025e0 0x7fd15c19c650 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 >> 172.21.15.16:0/1223365779 conn(0x7fd15c079710 msgr2=0x7fd15c109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 shutdown_connections 2024-09-21T02:58:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:19.880+0000 7fd163f68640 1 -- 172.21.15.16:0/1223365779 wait complete. 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1223365779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:20.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:20.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:20 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:20.526 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph --format=json-pretty osd lspools 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1223365779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.586 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:20 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:20.587 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: pgmap v251: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1223365779' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:20.588 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:20 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:58:21.488 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 -- 172.21.15.16:0/3586484320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64102240 msgr2=0x7fdd64102650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/3586484320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64102240 0x7fdd64102650 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fdd54009900 tx=0x7fdd5402a410 comp rx=0 tx=0).stop 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 -- 172.21.15.16:0/3586484320 shutdown_connections 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/3586484320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd641084d0 0x7fdd6410ed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/3586484320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd64107b80 0x7fdd64107f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/3586484320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64102240 0x7fdd64102650 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 -- 172.21.15.16:0/3586484320 >> 172.21.15.16:0/3586484320 conn(0x7fdd64079f90 msgr2=0x7fdd6407a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 -- 172.21.15.16:0/3586484320 shutdown_connections 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.083+0000 7fdd6a73b640 1 -- 172.21.15.16:0/3586484320 wait complete. 2024-09-21T02:58:22.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.084+0000 7fdd6a73b640 1 Processor -- start 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.084+0000 7fdd6a73b640 1 -- start start 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd64102240 0x7fdd640fedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd641084d0 0x7fdd640ff870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111910 con 0x7fdd64107b80 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111790 con 0x7fdd641084d0 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd64111a90 con 0x7fdd64102240 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51774/0 (socket says 172.21.15.16:51774) 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 -- 172.21.15.16:0/1824021302 learned_addr learned my addr 172.21.15.16:0/1824021302 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd63fff640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd64102240 0x7fdd640fedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 -- 172.21.15.16:0/1824021302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd64102240 msgr2=0x7fdd640fedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd64102240 0x7fdd640fedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd68cd8640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd641084d0 0x7fdd640ff870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 -- 172.21.15.16:0/1824021302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd641084d0 msgr2=0x7fdd640ff870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd641084d0 0x7fdd640ff870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 -- 172.21.15.16:0/1824021302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd641a8860 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd637fe640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d740 tx=0x7fdd5800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd58017070 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd5800de50 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd641a8b50 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd5800f620 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.085+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd641a90e0 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.086+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd64102650 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.088+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fdd5801a070 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.089+0000 7fdd617fa640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 0x7fdd4c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.089+0000 7fdd63fff640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 0x7fdd4c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.089+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7fdd580e6a80 con 0x7fdd64107b80 2024-09-21T02:58:22.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.089+0000 7fdd63fff640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 0x7fdd4c0b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000c00 tx=0x7fdd54031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:22.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.092+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdd580ad6c0 con 0x7fdd64107b80 2024-09-21T02:58:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.300+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7fdd6402e640 con 0x7fdd64107b80 2024-09-21T02:58:22.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.301+0000 7fdd617fa640 1 -- 172.21.15.16:0/1824021302 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v70) ==== 84+0+65 (secure 0 0 0) 0x7fdd580b4e20 con 0x7fdd64107b80 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout:[ 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout: "poolnum": 1, 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout: "poolname": ".mgr" 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T02:58:22.304 INFO:teuthology.orchestra.run.smithi016.stdout:] 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 msgr2=0x7fdd4c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 0x7fdd4c0b8af0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fdd54000c00 tx=0x7fdd54031400 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 msgr2=0x7fdd640ff330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fdd5800d740 tx=0x7fdd5800dc00 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 shutdown_connections 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd4c0b65e0 0x7fdd4c0b8af0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd64107b80 0x7fdd640ff330 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd641084d0 0x7fdd640ff870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 --2- 172.21.15.16:0/1824021302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd64102240 0x7fdd640fedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 >> 172.21.15.16:0/1824021302 conn(0x7fdd64079f90 msgr2=0x7fdd641013c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 shutdown_connections 2024-09-21T02:58:22.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:22.304+0000 7fdd6a73b640 1 -- 172.21.15.16:0/1824021302 wait complete. 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1824021302' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:58:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1824021302' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:58:22.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:22.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:22.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1824021302' entity='client.admin' cmd={"prefix": "osd lspools", "format": "json-pretty"} : dispatch 2024-09-21T02:58:23.088 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-09-21T02:58:23.088 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_metadata --pg_num_min 64 2024-09-21T02:58:23.362 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:23 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 -- 172.21.15.16:0/1160034999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc103b70 msgr2=0x7f28dc103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 --2- 172.21.15.16:0/1160034999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc103b70 0x7f28dc103f80 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f28d8009900 tx=0x7f28d802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 -- 172.21.15.16:0/1160034999 shutdown_connections 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 --2- 172.21.15.16:0/1160034999 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc109e00 0x7f28dc110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 --2- 172.21.15.16:0/1160034999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 0x7f28dc1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 --2- 172.21.15.16:0/1160034999 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc103b70 0x7f28dc103f80 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 -- 172.21.15.16:0/1160034999 >> 172.21.15.16:0/1160034999 conn(0x7f28dc0ff5e0 msgr2=0x7f28dc101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 -- 172.21.15.16:0/1160034999 shutdown_connections 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.858+0000 7f28e45d8640 1 -- 172.21.15.16:0/1160034999 wait complete. 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 Processor -- start 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 -- start start 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc103b70 0x7f28dc19a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 0x7f28dc19aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc113ac0 con 0x7f28dc109e00 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc113940 con 0x7f28dc103b70 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e45d8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28dc113c40 con 0x7f28dc1094b0 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e2b75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e2b75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51794/0 (socket says 172.21.15.16:51794) 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e2374640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc103b70 0x7f28dc19a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.859+0000 7f28e2b75640 1 -- 172.21.15.16:0/3528912893 learned_addr learned my addr 172.21.15.16:0/3528912893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e1b73640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 0x7f28dc19aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 -- 172.21.15.16:0/3528912893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 msgr2=0x7f28dc19aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 0x7f28dc19aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 -- 172.21.15.16:0/3528912893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc103b70 msgr2=0x7f28dc19a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc103b70 0x7f28dc19a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:23.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 -- 172.21.15.16:0/3528912893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28dc1a03a0 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e2b75640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f28d400cad0 tx=0x7f28d400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400ae20 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28d4005020 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28dc1a0690 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28d400f6b0 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.860+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28dc1a6eb0 con 0x7f28dc109e00 2024-09-21T02:58:23.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.862+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28dc104df0 con 0x7f28dc109e00 2024-09-21T02:58:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.863+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f28d40051d0 con 0x7f28dc109e00 2024-09-21T02:58:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.865+0000 7f28d37fe640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 0x7f28c80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.865+0000 7f28e2374640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 0x7f28c80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.865+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(70..70 src has 1..70) ==== 7342+0+0 (secure 0 0 0) 0x7f28d40e7070 con 0x7f28dc109e00 2024-09-21T02:58:23.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.865+0000 7f28e2374640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 0x7f28c80b8aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28d8031440 tx=0x7f28d8005e80 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:23.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:23.869+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f28d4014030 con 0x7f28dc109e00 2024-09-21T02:58:23.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:23.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:23.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:23 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:23.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:58:23.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:58:23.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:23 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:58:24.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.056+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7f28dc10bbd0 con 0x7f28dc109e00 2024-09-21T02:58:24.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.614+0000 7f28d37fe640 1 -- 172.21.15.16:0/3528912893 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v71) ==== 138+0+0 (secure 0 0 0) 0x7f28d40adc30 con 0x7f28dc109e00 2024-09-21T02:58:24.617 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' created 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 msgr2=0x7f28c80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 0x7f28c80b8aa0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f28d8031440 tx=0x7f28d8005e80 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 msgr2=0x7f28dc19fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f28d400cad0 tx=0x7f28d400cf90 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 shutdown_connections 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f28c80b6590 0x7f28c80b8aa0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28dc109e00 0x7f28dc19fd30 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28dc1094b0 0x7f28dc19aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 --2- 172.21.15.16:0/3528912893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28dc103b70 0x7f28dc19a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 >> 172.21.15.16:0/3528912893 conn(0x7f28dc0ff5e0 msgr2=0x7f28dc101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 shutdown_connections 2024-09-21T02:58:24.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:24.619+0000 7f28e45d8640 1 -- 172.21.15.16:0/3528912893 wait complete. 2024-09-21T02:58:24.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:24 smithi018 ceph-mon[34587]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:24.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:24 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:58:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:24 smithi123 ceph-mon[34645]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:24 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:58:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:24 smithi016 ceph-mon[31253]: pgmap v253: 1 pgs: 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:24 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} : dispatch 2024-09-21T02:58:25.381 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2024-09-21T02:58:25.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:25 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:58:25.624 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:25 smithi016 ceph-mon[31253]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:58:25.654 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:25.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:25 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:58:25.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:25 smithi018 ceph-mon[34587]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:58:25.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:25 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3528912893' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-09-21T02:58:25.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:25 smithi123 ceph-mon[34645]: osdmap e71: 12 total, 12 up, 12 in 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 -- 172.21.15.16:0/1798065579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 msgr2=0x7fbf4010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1798065579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4010f520 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009900 tx=0x7fbf3802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 -- 172.21.15.16:0/1798065579 shutdown_connections 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1798065579 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4010f520 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1798065579 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf401087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1798065579 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 0x7fbf40102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 -- 172.21.15.16:0/1798065579 >> 172.21.15.16:0/1798065579 conn(0x7fbf40079730 msgr2=0x7fbf40079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 -- 172.21.15.16:0/1798065579 shutdown_connections 2024-09-21T02:58:26.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.187+0000 7fbf44d13640 1 -- 172.21.15.16:0/1798065579 wait complete. 2024-09-21T02:58:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 Processor -- start 2024-09-21T02:58:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 -- start start 2024-09-21T02:58:26.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 0x7fbf4019b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf4019b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40112960 con 0x7fbf40108cf0 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbf401127e0 con 0x7fbf40102a60 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf44d13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf40112ae0 con 0x7fbf401083a0 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf3ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38756/0 (socket says 172.21.15.16:38756) 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.188+0000 7fbf3ed9d640 1 -- 172.21.15.16:0/1892530113 learned_addr learned my addr 172.21.15.16:0/1892530113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3e59c640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 0x7fbf4019b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3dd9b640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf4019b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 -- 172.21.15.16:0/1892530113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 msgr2=0x7fbf4019b750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf4019b750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 -- 172.21.15.16:0/1892530113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 msgr2=0x7fbf4019b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:26.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 0x7fbf4019b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 -- 172.21.15.16:0/1892530113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf401a64d0 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3dd9b640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf4019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf3ed9d640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009a20 tx=0x7fbf38004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3803f030 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf401a6760 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf38003aa0 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf38035070 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.189+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf401a6c40 con 0x7fbf40108cf0 2024-09-21T02:58:26.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.190+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf40102e70 con 0x7fbf40108cf0 2024-09-21T02:58:26.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.193+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fbf38042030 con 0x7fbf40108cf0 2024-09-21T02:58:26.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.194+0000 7fbf2f7fe640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 0x7fbf280b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.194+0000 7fbf3e59c640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 0x7fbf280b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.194+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(72..72 src has 1..72) ==== 7712+0+0 (secure 0 0 0) 0x7fbf38107c50 con 0x7fbf40108cf0 2024-09-21T02:58:26.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.194+0000 7fbf3e59c640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 0x7fbf280b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005c40 tx=0x7fbf3000a590 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:26.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.197+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf380ce6c0 con 0x7fbf40108cf0 2024-09-21T02:58:26.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.388+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7fbf4006b290 con 0x7fbf40108cf0 2024-09-21T02:58:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.628+0000 7fbf2f7fe640 1 -- 172.21.15.16:0/1892530113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v73) ==== 170+0+0 (secure 0 0 0) 0x7fbf380d5e20 con 0x7fbf40108cf0 2024-09-21T02:58:26.630 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' created 2024-09-21T02:58:26.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 msgr2=0x7fbf280b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 0x7fbf280b8a70 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fbf30005c40 tx=0x7fbf3000a590 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 msgr2=0x7fbf4019bc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fbf38009a20 tx=0x7fbf38004270 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 shutdown_connections 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf280b6560 0x7fbf280b8a70 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf40108cf0 0x7fbf4019bc90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf401083a0 0x7fbf4019b750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 --2- 172.21.15.16:0/1892530113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf40102a60 0x7fbf4019b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.632+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 >> 172.21.15.16:0/1892530113 conn(0x7fbf40079730 msgr2=0x7fbf40104370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.633+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 shutdown_connections 2024-09-21T02:58:26.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:26.633+0000 7fbf44d13640 1 -- 172.21.15.16:0/1892530113 wait complete. 2024-09-21T02:58:26.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:26 smithi018 ceph-mon[34587]: pgmap v255: 33 pgs: 20 creating+peering, 12 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:26.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:26 smithi018 ceph-mon[34587]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:58:26.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:26 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:58:26.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:26 smithi123 ceph-mon[34645]: pgmap v255: 33 pgs: 20 creating+peering, 12 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:26.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:26 smithi123 ceph-mon[34645]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:58:26.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:26 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:58:26.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:26 smithi016 ceph-mon[31253]: pgmap v255: 33 pgs: 20 creating+peering, 12 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:26.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:26 smithi016 ceph-mon[31253]: osdmap e72: 12 total, 12 up, 12 in 2024-09-21T02:58:26.986 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd={"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} : dispatch 2024-09-21T02:58:27.330 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs new cephfs cephfs_metadata cephfs_data 2024-09-21T02:58:27.599 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:27.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:27 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:58:27.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:27 smithi016 ceph-mon[31253]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:58:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:27 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:58:27.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:27 smithi018 ceph-mon[34587]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:58:27.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:27 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1892530113' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-09-21T02:58:27.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:27 smithi123 ceph-mon[34645]: osdmap e73: 12 total, 12 up, 12 in 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 -- 172.21.15.16:0/4027186573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c1083d0 msgr2=0x7f010c1087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 --2- 172.21.15.16:0/4027186573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c1083d0 0x7f010c1087e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f00fc009900 tx=0x7f00fc02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 -- 172.21.15.16:0/4027186573 shutdown_connections 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 --2- 172.21.15.16:0/4027186573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c1083d0 0x7f010c1087e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 --2- 172.21.15.16:0/4027186573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c108d20 0x7f010c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 --2- 172.21.15.16:0/4027186573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c102ae0 0x7f010c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 -- 172.21.15.16:0/4027186573 >> 172.21.15.16:0/4027186573 conn(0x7f010c079760 msgr2=0x7f010c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 -- 172.21.15.16:0/4027186573 shutdown_connections 2024-09-21T02:58:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.126+0000 7f01144a6640 1 -- 172.21.15.16:0/4027186573 wait complete. 2024-09-21T02:58:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 Processor -- start 2024-09-21T02:58:28.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 -- start start 2024-09-21T02:58:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102ae0 0x7f010c1990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c1083d0 0x7f010c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f010c1129e0 con 0x7f010c108d20 2024-09-21T02:58:28.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112860 con 0x7f010c102ae0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f01144a6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f010c112b60 con 0x7f010c1083d0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f0112a43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f0112a43640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38776/0 (socket says 172.21.15.16:38776) 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f0112a43640 1 -- 172.21.15.16:0/1497004124 learned_addr learned my addr 172.21.15.16:0/1497004124 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f0112242640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102ae0 0x7f010c1990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.127+0000 7f0111a41640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c1083d0 0x7f010c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 -- 172.21.15.16:0/1497004124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c1083d0 msgr2=0x7f010c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c1083d0 0x7f010c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 -- 172.21.15.16:0/1497004124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102ae0 msgr2=0x7f010c1990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102ae0 0x7f010c1990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 -- 172.21.15.16:0/1497004124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f010c1a8750 con 0x7f010c108d20 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f0112a43640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f010400eaa0 tx=0x7f010400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f010400cdb0 con 0x7f010c108d20 2024-09-21T02:58:28.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0104005020 con 0x7f010c108d20 2024-09-21T02:58:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01040116d0 con 0x7f010c108d20 2024-09-21T02:58:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f010c1a8a40 con 0x7f010c108d20 2024-09-21T02:58:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.128+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f010c1a8f00 con 0x7f010c108d20 2024-09-21T02:58:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.129+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f010c103d10 con 0x7f010c108d20 2024-09-21T02:58:28.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.131+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f01040051d0 con 0x7f010c108d20 2024-09-21T02:58:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.132+0000 7f01037fe640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 0x7f00f80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.132+0000 7f0112242640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 0x7f00f80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:28.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.132+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 8094+0+0 (secure 0 0 0) 0x7f01040e7840 con 0x7f010c108d20 2024-09-21T02:58:28.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.132+0000 7f0112242640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 0x7f00f80b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f010c1060a0 tx=0x7f010800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:28.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.135+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0104016030 con 0x7f010c108d20 2024-09-21T02:58:28.365 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.363+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f010c02e640 con 0x7f010c108d20 2024-09-21T02:58:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.635+0000 7f01037fe640 1 -- 172.21.15.16:0/1497004124 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f01040b24f0 con 0x7f010c108d20 2024-09-21T02:58:28.637 INFO:teuthology.orchestra.run.smithi016.stderr:new fs with metadata pool 2 and data pool 3 2024-09-21T02:58:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.638+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 msgr2=0x7f00f80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 0x7f00f80b8a70 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f010c1060a0 tx=0x7f010800b390 comp rx=0 tx=0).stop 2024-09-21T02:58:28.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 msgr2=0x7f010c199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f010400eaa0 tx=0x7f010400ef60 comp rx=0 tx=0).stop 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 shutdown_connections 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00f80b6560 0x7f00f80b8a70 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f010c108d20 0x7f010c199b20 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f010c1083d0 0x7f010c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 --2- 172.21.15.16:0/1497004124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f010c102ae0 0x7f010c1990a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 >> 172.21.15.16:0/1497004124 conn(0x7f010c079760 msgr2=0x7f010c104400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 shutdown_connections 2024-09-21T02:58:28.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:28.639+0000 7f01144a6640 1 -- 172.21.15.16:0/1497004124 wait complete. 2024-09-21T02:58:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:28 smithi018 ceph-mon[34587]: pgmap v258: 97 pgs: 30 creating+peering, 66 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:28 smithi018 ceph-mon[34587]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:58:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:28 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:58:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:28 smithi018 ceph-mon[34587]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:58:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:28 smithi123 ceph-mon[34645]: pgmap v258: 97 pgs: 30 creating+peering, 66 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:28 smithi123 ceph-mon[34645]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:58:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:58:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:28 smithi123 ceph-mon[34645]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:58:29.001 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:28 smithi016 ceph-mon[31253]: pgmap v258: 97 pgs: 30 creating+peering, 66 unknown, 1 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:29.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:28 smithi016 ceph-mon[31253]: osdmap e74: 12 total, 12 up, 12 in 2024-09-21T02:58:29.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:28 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd={"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} : dispatch 2024-09-21T02:58:29.002 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:28 smithi016 ceph-mon[31253]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-09-21T02:58:29.355 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:58:29.612 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:29.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:29 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:58:29.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:29 smithi016 ceph-mon[31253]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:58:29.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:29 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:58:29.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:29 smithi016 ceph-mon[31253]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:58:29.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:29 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:58:29.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:29 smithi018 ceph-mon[34587]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:58:29.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:29 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:58:29.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:29 smithi018 ceph-mon[34587]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:58:29.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:29 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1497004124' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-09-21T02:58:29.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:29 smithi123 ceph-mon[34645]: osdmap e75: 12 total, 12 up, 12 in 2024-09-21T02:58:29.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:29 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-09-21T02:58:29.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:29 smithi123 ceph-mon[34645]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-09-21T02:58:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.088+0000 7f3fe4e91640 1 -- 172.21.15.16:0/133585325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 msgr2=0x7f3fe010f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.088+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/133585325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe010f790 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8009ca0 tx=0x7f3fd802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 -- 172.21.15.16:0/133585325 shutdown_connections 2024-09-21T02:58:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/133585325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe010f790 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/133585325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe0108610 0x7f3fe0108a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/133585325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe01031d0 0x7f3fe01035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 -- 172.21.15.16:0/133585325 >> 172.21.15.16:0/133585325 conn(0x7f3fe0079e40 msgr2=0x7f3fe007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:30.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 -- 172.21.15.16:0/133585325 shutdown_connections 2024-09-21T02:58:30.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.089+0000 7f3fe4e91640 1 -- 172.21.15.16:0/133585325 wait complete. 2024-09-21T02:58:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 Processor -- start 2024-09-21T02:58:30.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 -- start start 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe01031d0 0x7f3fe019b5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe0108610 0x7f3fe019baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0112c20 con 0x7f3fe0108f60 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0112aa0 con 0x7f3fe01031d0 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fe4e91640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fe0112da0 con 0x7f3fe0108610 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fde59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe01031d0 0x7f3fe019b5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fded9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38790/0 (socket says 172.21.15.16:38790) 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fded9d640 1 -- 172.21.15.16:0/7561930 learned_addr learned my addr 172.21.15.16:0/7561930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.090+0000 7f3fddd9b640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe0108610 0x7f3fe019baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 -- 172.21.15.16:0/7561930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe0108610 msgr2=0x7f3fe019baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe0108610 0x7f3fe019baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 -- 172.21.15.16:0/7561930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe01031d0 msgr2=0x7f3fe019b5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe01031d0 0x7f3fe019b5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 -- 172.21.15.16:0/7561930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fe01a6840 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fded9d640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8005cc0 tx=0x7f3fd802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd8003bf0 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fd8003d90 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fe01a6ad0 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fd802fbc0 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.091+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fe01a7010 con 0x7f3fe0108f60 2024-09-21T02:58:30.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.092+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fe01035e0 con 0x7f3fe0108f60 2024-09-21T02:58:30.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.094+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f3fd802fd60 con 0x7f3fe0108f60 2024-09-21T02:58:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.095+0000 7f3fcf7fe640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:30.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.095+0000 7f3fde59c640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.095+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f3fd8108780 con 0x7f3fe0108f60 2024-09-21T02:58:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.096+0000 7f3fde59c640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0102990 tx=0x7f3fd000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.098+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fd80cf030 con 0x7f3fe0108f60 2024-09-21T02:58:30.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.289+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fe0102240 con 0x7f3fe0108f60 2024-09-21T02:58:30.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.291+0000 7f3fcf7fe640 1 -- 172.21.15.16:0/7561930 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22582 (secure 0 0 0) 0x7f3fd80098b0 con 0x7f3fe0108f60 2024-09-21T02:58:30.293 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:30.293 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:28.365049+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 msgr2=0x7f3fc80b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0102990 tx=0x7f3fd000b390 comp rx=0 tx=0).stop 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 msgr2=0x7f3fe019c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3fd8005cc0 tx=0x7f3fd802f380 comp rx=0 tx=0).stop 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 shutdown_connections 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fc80b6590 0x7f3fc80b8aa0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fe0108f60 0x7f3fe019c030 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fe0108610 0x7f3fe019baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 --2- 172.21.15.16:0/7561930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fe01031d0 0x7f3fe019b5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 >> 172.21.15.16:0/7561930 conn(0x7f3fe0079e40 msgr2=0x7f3fe01045e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 shutdown_connections 2024-09-21T02:58:30.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:30.293+0000 7f3fe4e91640 1 -- 172.21.15.16:0/7561930 wait complete. 2024-09-21T02:58:30.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:30 smithi016 ceph-mon[31253]: pgmap v261: 97 pgs: 30 creating+peering, 41 unknown, 26 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:30.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:30 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:active 2024-09-21T02:58:30.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:30 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:30 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.905 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/7561930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:30.907 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:58:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:30 smithi018 ceph-mon[34587]: pgmap v261: 97 pgs: 30 creating+peering, 41 unknown, 26 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:30 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:active 2024-09-21T02:58:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:30 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:30 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/7561930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:30.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:30 smithi123 ceph-mon[34645]: pgmap v261: 97 pgs: 30 creating+peering, 41 unknown, 26 active+clean; 577 KiB data, 372 MiB used, 1.0 TiB / 1.0 TiB avail 2024-09-21T02:58:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:30 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] up:active 2024-09-21T02:58:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:30 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:30 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/7561930' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:31.163 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 -- 172.21.15.16:0/3727046071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f18108cf0 msgr2=0x7f2f1810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/3727046071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f18108cf0 0x7f2f1810f520 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2f10009ca0 tx=0x7f2f1002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 -- 172.21.15.16:0/3727046071 shutdown_connections 2024-09-21T02:58:31.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/3727046071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f18108cf0 0x7f2f1810f520 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/3727046071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f181083a0 0x7f2f181087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.655+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/3727046071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18102a60 0x7f2f18102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.656+0000 7f2f1fef2640 1 -- 172.21.15.16:0/3727046071 >> 172.21.15.16:0/3727046071 conn(0x7f2f18079730 msgr2=0x7f2f18079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.656+0000 7f2f1fef2640 1 -- 172.21.15.16:0/3727046071 shutdown_connections 2024-09-21T02:58:31.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.656+0000 7f2f1fef2640 1 -- 172.21.15.16:0/3727046071 wait complete. 2024-09-21T02:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.656+0000 7f2f1fef2640 1 Processor -- start 2024-09-21T02:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.656+0000 7f2f1fef2640 1 -- start start 2024-09-21T02:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 0x7f2f1819b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:31.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18108cf0 0x7f2f1819bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18112960 con 0x7f2f181083a0 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2f181127e0 con 0x7f2f18102a60 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1fef2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2f18112ae0 con 0x7f2f18108cf0 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1dc8e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 0x7f2f1819b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38808/0 (socket says 172.21.15.16:38808) 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 -- 172.21.15.16:0/1087512848 learned_addr learned my addr 172.21.15.16:0/1087512848 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1e48f640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18108cf0 0x7f2f1819bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 -- 172.21.15.16:0/1087512848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18108cf0 msgr2=0x7f2f1819bc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18108cf0 0x7f2f1819bc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 -- 172.21.15.16:0/1087512848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 msgr2=0x7f2f1819b210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 0x7f2f1819b210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 -- 172.21.15.16:0/1087512848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2f181a6520 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1dc8e640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 0x7f2f1819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.657+0000 7f2f1d48d640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e9c0 tx=0x7f2f0800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.658+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f0800cd80 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.658+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f080054d0 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.658+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2f181a6810 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.658+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f08010430 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.658+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2f181a6da0 con 0x7f2f181083a0 2024-09-21T02:58:31.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.659+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2f18102e70 con 0x7f2f181083a0 2024-09-21T02:58:31.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.660+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f2f08004fb0 con 0x7f2f181083a0 2024-09-21T02:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.661+0000 7f2f0effd640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 0x7f2f040b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.661+0000 7f2f1dc8e640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 0x7f2f040b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.661+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f2f080e78e0 con 0x7f2f181083a0 2024-09-21T02:58:31.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.662+0000 7f2f1dc8e640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 0x7f2f040b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f1806ba20 tx=0x7f2f14005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:31.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.664+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f08056710 con 0x7f2f181083a0 2024-09-21T02:58:31.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.853+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f2f1806b2d0 con 0x7f2f181083a0 2024-09-21T02:58:31.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.854+0000 7f2f0effd640 1 -- 172.21.15.16:0/1087512848 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22582 (secure 0 0 0) 0x7f2f1806b2d0 con 0x7f2f181083a0 2024-09-21T02:58:31.856 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:31.856 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:28.365049+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 msgr2=0x7f2f040b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 0x7f2f040b8af0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f2f1806ba20 tx=0x7f2f14005f90 comp rx=0 tx=0).stop 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 msgr2=0x7f2f1819b750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2f0800e9c0 tx=0x7f2f0800ee80 comp rx=0 tx=0).stop 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 shutdown_connections 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f040b65e0 0x7f2f040b8af0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2f181083a0 0x7f2f1819b750 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2f18108cf0 0x7f2f1819bc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 --2- 172.21.15.16:0/1087512848 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2f18102a60 0x7f2f1819b210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 >> 172.21.15.16:0/1087512848 conn(0x7f2f18079730 msgr2=0x7f2f18104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 shutdown_connections 2024-09-21T02:58:31.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:31.857+0000 7f2f1fef2640 1 -- 172.21.15.16:0/1087512848 wait complete. 2024-09-21T02:58:32.492 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs set cephfs standby_count_wanted 0 2024-09-21T02:58:32.748 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:32.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:32 smithi016 ceph-mon[31253]: pgmap v262: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T02:58:32.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:32 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1087512848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:32.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:32 smithi018 ceph-mon[34587]: pgmap v262: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T02:58:32.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:32 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1087512848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:32 smithi123 ceph-mon[34645]: pgmap v262: 97 pgs: 7 creating+peering, 90 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 7 op/s 2024-09-21T02:58:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:32 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1087512848' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 -- 172.21.15.16:0/3999085775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30102a60 msgr2=0x7f4b30102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 --2- 172.21.15.16:0/3999085775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30102a60 0x7f4b30102e70 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c009900 tx=0x7f4b2c02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 -- 172.21.15.16:0/3999085775 shutdown_connections 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 --2- 172.21.15.16:0/3999085775 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30108cf0 0x7f4b3010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 --2- 172.21.15.16:0/3999085775 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 0x7f4b301087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 --2- 172.21.15.16:0/3999085775 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30102a60 0x7f4b30102e70 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 -- 172.21.15.16:0/3999085775 >> 172.21.15.16:0/3999085775 conn(0x7f4b30079730 msgr2=0x7f4b30079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 -- 172.21.15.16:0/3999085775 shutdown_connections 2024-09-21T02:58:33.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.218+0000 7f4b386b7640 1 -- 172.21.15.16:0/3999085775 wait complete. 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 Processor -- start 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 -- start start 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30102a60 0x7f4b3019a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 0x7f4b3019a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30113a80 con 0x7f4b30108cf0 2024-09-21T02:58:33.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30113900 con 0x7f4b301083a0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b386b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4b30113c00 con 0x7f4b30102a60 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b36c54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b36453640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30102a60 0x7f4b3019a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b36c54640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:38834/0 (socket says 172.21.15.16:38834) 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.219+0000 7f4b36c54640 1 -- 172.21.15.16:0/2513348982 learned_addr learned my addr 172.21.15.16:0/2513348982 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b35c52640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 0x7f4b3019a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 -- 172.21.15.16:0/2513348982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30102a60 msgr2=0x7f4b3019a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30102a60 0x7f4b3019a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 -- 172.21.15.16:0/2513348982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 msgr2=0x7f4b3019a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 0x7f4b3019a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 -- 172.21.15.16:0/2513348982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b301a03c0 con 0x7f4b30108cf0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b36c54640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800eaa0 tx=0x7f4b2800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b2800cdb0 con 0x7f4b30108cf0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b301a06b0 con 0x7f4b30108cf0 2024-09-21T02:58:33.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b28005020 con 0x7f4b30108cf0 2024-09-21T02:58:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b280116d0 con 0x7f4b30108cf0 2024-09-21T02:58:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.220+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b301a6e10 con 0x7f4b30108cf0 2024-09-21T02:58:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.221+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b30102e70 con 0x7f4b30108cf0 2024-09-21T02:58:33.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.223+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7f4b280051d0 con 0x7f4b30108cf0 2024-09-21T02:58:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.223+0000 7f4b277fe640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 0x7f4b1c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.224+0000 7f4b36453640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 0x7f4b1c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.224+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f4b280e78c0 con 0x7f4b30108cf0 2024-09-21T02:58:33.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.224+0000 7f4b36453640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 0x7f4b1c0b8af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c031440 tx=0x7f4b2c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:33.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.227+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4b28016030 con 0x7f4b30108cf0 2024-09-21T02:58:33.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.453+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f4b3019b580 con 0x7f4b30108cf0 2024-09-21T02:58:33.655 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 02:58:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: 2024-09-21T02:58:33.356+0000 7fb47792d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-09-21T02:58:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.658+0000 7f4b277fe640 1 -- 172.21.15.16:0/2513348982 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v17) ==== 118+0+0 (secure 0 0 0) 0x7f4b280b24f0 con 0x7f4b30108cf0 2024-09-21T02:58:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 msgr2=0x7f4b1c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 0x7f4b1c0b8af0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4b2c031440 tx=0x7f4b2c009560 comp rx=0 tx=0).stop 2024-09-21T02:58:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 msgr2=0x7f4b3019fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f4b2800eaa0 tx=0x7f4b2800ef60 comp rx=0 tx=0).stop 2024-09-21T02:58:33.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 shutdown_connections 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4b1c0b65e0 0x7f4b1c0b8af0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4b30108cf0 0x7f4b3019fc90 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4b301083a0 0x7f4b3019a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 --2- 172.21.15.16:0/2513348982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4b30102a60 0x7f4b3019a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.661+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 >> 172.21.15.16:0/2513348982 conn(0x7f4b30079730 msgr2=0x7f4b301013d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.662+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 shutdown_connections 2024-09-21T02:58:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:33.662+0000 7f4b386b7640 1 -- 172.21.15.16:0/2513348982 wait complete. 2024-09-21T02:58:33.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:58:33.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:58:33.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} : dispatch 2024-09-21T02:58:34.338 DEBUG:tasks.cephfs.filesystem:fs_config: {'max_mds': 1, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2}} 2024-09-21T02:58:34.338 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs set cephfs allow_standby_replay true 2024-09-21T02:58:34.600 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:34 smithi018 ceph-mon[34587]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T02:58:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:34 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:58:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:34 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:34.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:34 smithi016 ceph-mon[31253]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T02:58:34.966 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:34 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:58:34.967 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:34 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:34 smithi123 ceph-mon[34645]: pgmap v263: 97 pgs: 97 active+clean; 579 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 2.1 KiB/s wr, 6 op/s 2024-09-21T02:58:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:34 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2513348982' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-09-21T02:58:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:34 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 -- 172.21.15.16:0/491376714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d0103b60 msgr2=0x7fc6d0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:35.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 --2- 172.21.15.16:0/491376714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d0103b60 0x7fc6d0103f70 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009900 tx=0x7fc6c002a410 comp rx=0 tx=0).stop 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 -- 172.21.15.16:0/491376714 shutdown_connections 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 --2- 172.21.15.16:0/491376714 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 --2- 172.21.15.16:0/491376714 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d01094a0 0x7fc6d01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 --2- 172.21.15.16:0/491376714 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d0103b60 0x7fc6d0103f70 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 -- 172.21.15.16:0/491376714 >> 172.21.15.16:0/491376714 conn(0x7fc6d007b850 msgr2=0x7fc6d007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 -- 172.21.15.16:0/491376714 shutdown_connections 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.059+0000 7fc6d522f640 1 -- 172.21.15.16:0/491376714 wait complete. 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.060+0000 7fc6d522f640 1 Processor -- start 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.060+0000 7fc6d522f640 1 -- start start 2024-09-21T02:58:35.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113ab0 con 0x7fc6d01094a0 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113930 con 0x7fc6d0103b60 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6d522f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc6d0113c30 con 0x7fc6d0109df0 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60618/0 (socket says 172.21.15.16:60618) 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ced9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 -- 172.21.15.16:0/1551840802 learned_addr learned my addr 172.21.15.16:0/1551840802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6cf59e640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 -- 172.21.15.16:0/1551840802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 msgr2=0x7fc6d019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:35.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 -- 172.21.15.16:0/1551840802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 msgr2=0x7fc6d019a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 -- 172.21.15.16:0/1551840802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc6d01a03f0 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ced9d640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6cf59e640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.061+0000 7fc6ce59c640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400e9f0 tx=0x7fc6c400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.062+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c400ccc0 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.062+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc6c40108e0 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.062+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc6d01a06e0 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.062+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4018860 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.062+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc6d01a6e40 con 0x7fc6d01094a0 2024-09-21T02:58:35.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.063+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc6d0103f70 con 0x7fc6d01094a0 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.064+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fc6c4010430 con 0x7fc6d01094a0 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.065+0000 7fc6b7fff640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.065+0000 7fc6ced9d640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.065+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fc6c40e7b50 con 0x7fc6d01094a0 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.066+0000 7fc6ced9d640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 0x7fc6b80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0031440 tx=0x7fc6c0032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:35.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.068+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc6c40ae400 con 0x7fc6d01094a0 2024-09-21T02:58:35.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.296+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7fc6d019b5b0 con 0x7fc6d01094a0 2024-09-21T02:58:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.668+0000 7fc6b7fff640 1 -- 172.21.15.16:0/1551840802 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v18) ==== 121+0+0 (secure 0 0 0) 0x7fc6d019b5b0 con 0x7fc6d01094a0 2024-09-21T02:58:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.670+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 msgr2=0x7fc6b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.670+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 0x7fc6b80b8a70 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0031440 tx=0x7fc6c0032ed0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.670+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 msgr2=0x7fc6d019aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.670+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400e9f0 tx=0x7fc6c400eeb0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 shutdown_connections 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc6b80b6560 0x7fc6b80b8a70 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc6d01094a0 0x7fc6d019aa20 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6d0109df0 0x7fc6d019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 --2- 172.21.15.16:0/1551840802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc6d0103b60 0x7fc6d019a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 >> 172.21.15.16:0/1551840802 conn(0x7fc6d007b850 msgr2=0x7fc6d0107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 shutdown_connections 2024-09-21T02:58:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:35.671+0000 7fc6d522f640 1 -- 172.21.15.16:0/1551840802 wait complete. 2024-09-21T02:58:35.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:35 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:58:35.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:35 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:58:36.003 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:35 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} : dispatch 2024-09-21T02:58:36.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs set cephfs session_timeout 300 2024-09-21T02:58:36.587 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:36 smithi018 ceph-mon[34587]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:58:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:36 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:58:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:36 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:36 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:36 smithi123 ceph-mon[34645]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:58:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:36 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:58:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:36 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:36 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:36 smithi016 ceph-mon[31253]: pgmap v264: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s wr, 5 op/s 2024-09-21T02:58:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:36 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1551840802' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-09-21T02:58:37.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:36 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-09-21T02:58:37.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:36 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 -- 172.21.15.16:0/2183884878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c102a80 msgr2=0x7fa66c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 --2- 172.21.15.16:0/2183884878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c102a80 0x7fa66c102e90 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fa65c009900 tx=0x7fa65c02a3f0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 -- 172.21.15.16:0/2183884878 shutdown_connections 2024-09-21T02:58:37.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 --2- 172.21.15.16:0/2183884878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c108d10 0x7fa66c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 --2- 172.21.15.16:0/2183884878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 0x7fa66c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 --2- 172.21.15.16:0/2183884878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c102a80 0x7fa66c102e90 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 -- 172.21.15.16:0/2183884878 >> 172.21.15.16:0/2183884878 conn(0x7fa66c079740 msgr2=0x7fa66c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 -- 172.21.15.16:0/2183884878 shutdown_connections 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.244+0000 7fa672b37640 1 -- 172.21.15.16:0/2183884878 wait complete. 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.245+0000 7fa672b37640 1 Processor -- start 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.245+0000 7fa672b37640 1 -- start start 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c102a80 0x7fa66c19b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 0x7fa66c19b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c1129e0 con 0x7fa66c108d10 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c112860 con 0x7fa66c1083c0 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa672b37640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa66c112b60 con 0x7fa66c102a80 2024-09-21T02:58:37.249 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60632/0 (socket says 172.21.15.16:60632) 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 -- 172.21.15.16:0/3907412835 learned_addr learned my addr 172.21.15.16:0/3907412835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6708d3640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c102a80 0x7fa66c19b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa66bfff640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 0x7fa66c19b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 -- 172.21.15.16:0/3907412835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c102a80 msgr2=0x7fa66c19b3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c102a80 0x7fa66c19b3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 -- 172.21.15.16:0/3907412835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 msgr2=0x7fa66c19b8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 0x7fa66c19b8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 -- 172.21.15.16:0/3907412835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa66c1a65b0 con 0x7fa66c108d10 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.246+0000 7fa6710d4640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c970 tx=0x7fa66400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.247+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664007bb0 con 0x7fa66c108d10 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.247+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa664007d50 con 0x7fa66c108d10 2024-09-21T02:58:37.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.247+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa66c1a68a0 con 0x7fa66c108d10 2024-09-21T02:58:37.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.247+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa664018770 con 0x7fa66c108d10 2024-09-21T02:58:37.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.247+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa66c1a6de0 con 0x7fa66c108d10 2024-09-21T02:58:37.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.248+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa66c102e90 con 0x7fa66c108d10 2024-09-21T02:58:37.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.250+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fa66400f990 con 0x7fa66c108d10 2024-09-21T02:58:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.251+0000 7fa669ffb640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 0x7fa6580b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.252+0000 7fa6708d3640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 0x7fa6580b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.252+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fa6640e7390 con 0x7fa66c108d10 2024-09-21T02:58:37.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.252+0000 7fa6708d3640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 0x7fa6580b8b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa65c000c00 tx=0x7fa65c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:37.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.255+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa664014030 con 0x7fa66c108d10 2024-09-21T02:58:37.482 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.481+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7fa66c02e640 con 0x7fa66c108d10 2024-09-21T02:58:37.679 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.678+0000 7fa669ffb640 1 -- 172.21.15.16:0/3907412835 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v20) ==== 115+0+0 (secure 0 0 0) 0x7fa6640add50 con 0x7fa66c108d10 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 msgr2=0x7fa6580b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 0x7fa6580b8b20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa65c000c00 tx=0x7fa65c031400 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 msgr2=0x7fa66c19be30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fa66400c970 tx=0x7fa66400ce30 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 shutdown_connections 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa6580b6610 0x7fa6580b8b20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa66c108d10 0x7fa66c19be30 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa66c1083c0 0x7fa66c19b8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 --2- 172.21.15.16:0/3907412835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa66c102a80 0x7fa66c19b3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 >> 172.21.15.16:0/3907412835 conn(0x7fa66c079740 msgr2=0x7fa66c1014a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 shutdown_connections 2024-09-21T02:58:37.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:37.681+0000 7fa672b37640 1 -- 172.21.15.16:0/3907412835 wait complete. 2024-09-21T02:58:37.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:37 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:58:37.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:37 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:58:37.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:37 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd={"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} : dispatch 2024-09-21T02:58:38.284 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2} subvols for filesystem 'cephfs' 2024-09-21T02:58:38.284 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs subvolumegroup create cephfs qa 2024-09-21T02:58:38.545 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:38 smithi018 ceph-mon[34587]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T02:58:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:38 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:58:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:38 smithi018 ceph-mon[34587]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:38.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:38 smithi016 ceph-mon[31253]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T02:58:38.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:38 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:58:38.950 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:38 smithi016 ceph-mon[31253]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:38 smithi123 ceph-mon[34645]: pgmap v265: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 4 op/s 2024-09-21T02:58:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:38 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3907412835' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-09-21T02:58:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:38 smithi123 ceph-mon[34645]: fsmap cephfs:1 {0=b=up:active} 1 up:standby-replay 10 up:standby 2024-09-21T02:58:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 -- 172.21.15.16:0/1203705138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 msgr2=0x7fe7d010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:39.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 --2- 172.21.15.16:0/1203705138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d010f520 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8009900 tx=0x7fe7c802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 -- 172.21.15.16:0/1203705138 shutdown_connections 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 --2- 172.21.15.16:0/1203705138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d010f520 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 --2- 172.21.15.16:0/1203705138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 --2- 172.21.15.16:0/1203705138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 0x7fe7d0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 -- 172.21.15.16:0/1203705138 >> 172.21.15.16:0/1203705138 conn(0x7fe7d0079730 msgr2=0x7fe7d0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 -- 172.21.15.16:0/1203705138 shutdown_connections 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.044+0000 7fe7d5503640 1 -- 172.21.15.16:0/1203705138 wait complete. 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 Processor -- start 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 -- start start 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 0x7fe7d01050e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d0105620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d01129c0 con 0x7fe7d0108cf0 2024-09-21T02:58:39.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112840 con 0x7fe7d01083a0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7d5503640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7d0112b40 con 0x7fe7d0102a60 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7cf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60642/0 (socket says 172.21.15.16:60642) 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7cf7fe640 1 -- 172.21.15.16:0/247167413 learned_addr learned my addr 172.21.15.16:0/247167413 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7ceffd640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 0x7fe7d01050e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.045+0000 7fe7ce7fc640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d0105620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 -- 172.21.15.16:0/247167413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 msgr2=0x7fe7d01050e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 0x7fe7d01050e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 -- 172.21.15.16:0/247167413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 msgr2=0x7fe7d0105620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d0105620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 -- 172.21.15.16:0/247167413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7d01a8670 con 0x7fe7d0108cf0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7ce7fc640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7cf7fe640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006010 tx=0x7fe7c8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c803f030 con 0x7fe7d0108cf0 2024-09-21T02:58:39.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7d01a8900 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7c8003aa0 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7c8035070 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.046+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7d01a8eb0 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.047+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7d0102e70 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.049+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fe7c8042030 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.049+0000 7fe7c7fff640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 0x7fe7bc0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.050+0000 7fe7ceffd640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 0x7fe7bc0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.050+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fe7c8107ff0 con 0x7fe7d0108cf0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.050+0000 7fe7ceffd640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 0x7fe7bc0b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0106b30 tx=0x7fe7c000d040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:39.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.053+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe7c80ce880 con 0x7fe7d0108cf0 2024-09-21T02:58:39.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.265+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fe7d0063800 con 0x7fe7bc0b6560 2024-09-21T02:58:39.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.287+0000 7fe7c7fff640 1 -- 172.21.15.16:0/247167413 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fe7d0063800 con 0x7fe7bc0b6560 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 msgr2=0x7fe7bc0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 0x7fe7bc0b8a70 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe7d0106b30 tx=0x7fe7c000d040 comp rx=0 tx=0).stop 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 msgr2=0x7fe7d0105b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fe7c8006010 tx=0x7fe7c8004270 comp rx=0 tx=0).stop 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 shutdown_connections 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe7bc0b6560 0x7fe7bc0b8a70 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7d0108cf0 0x7fe7d0105b60 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7d01083a0 0x7fe7d0105620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 --2- 172.21.15.16:0/247167413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe7d0102a60 0x7fe7d01050e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 >> 172.21.15.16:0/247167413 conn(0x7fe7d0079730 msgr2=0x7fe7d006b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 shutdown_connections 2024-09-21T02:58:39.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:39.289+0000 7fe7d5503640 1 -- 172.21.15.16:0/247167413 wait complete. 2024-09-21T02:58:39.693 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:58:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:58:39.271+0000 7f4f99741640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:58:39.693 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:58:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:58:39.271+0000 7f4f99741640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:58:39.693 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:58:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:58:39.271+0000 7f4f99741640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:58:39.693 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:58:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:58:39.271+0000 7f4f99741640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:58:39.693 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 02:58:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T02:58:39.271+0000 7f4f99741640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T02:58:39.904 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_0 --group_name qa 2024-09-21T02:58:40.165 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 -- 172.21.15.16:0/1780488359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102a60 msgr2=0x7fdfe4102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 --2- 172.21.15.16:0/1780488359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102a60 0x7fdfe4102e70 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4009900 tx=0x7fdfd402a410 comp rx=0 tx=0).stop 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 -- 172.21.15.16:0/1780488359 shutdown_connections 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 --2- 172.21.15.16:0/1780488359 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe4108cf0 0x7fdfe410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 --2- 172.21.15.16:0/1780488359 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe41083a0 0x7fdfe41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 --2- 172.21.15.16:0/1780488359 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4102a60 0x7fdfe4102e70 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 -- 172.21.15.16:0/1780488359 >> 172.21.15.16:0/1780488359 conn(0x7fdfe4079730 msgr2=0x7fdfe4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 -- 172.21.15.16:0/1780488359 shutdown_connections 2024-09-21T02:58:40.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.615+0000 7fdfe8928640 1 -- 172.21.15.16:0/1780488359 wait complete. 2024-09-21T02:58:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.616+0000 7fdfe8928640 1 Processor -- start 2024-09-21T02:58:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.616+0000 7fdfe8928640 1 -- start start 2024-09-21T02:58:40.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 0x7fdfe419a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe41083a0 0x7fdfe419a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4113a80 con 0x7fdfe4108cf0 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4113900 con 0x7fdfe41083a0 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe8928640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdfe4113c00 con 0x7fdfe4102a60 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60670/0 (socket says 172.21.15.16:60670) 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 -- 172.21.15.16:0/4031822677 learned_addr learned my addr 172.21.15.16:0/4031822677 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe259c640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 0x7fdfe419a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe1d9b640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe41083a0 0x7fdfe419a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 -- 172.21.15.16:0/4031822677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 msgr2=0x7fdfe419a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:40.620 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 0x7fdfe419a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 -- 172.21.15.16:0/4031822677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe41083a0 msgr2=0x7fdfe419a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe41083a0 0x7fdfe419a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 -- 172.21.15.16:0/4031822677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdfe41a03c0 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe259c640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 0x7fdfe419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.617+0000 7fdfe2d9d640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00eaa0 tx=0x7fdfdc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.618+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc00cdb0 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.618+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdfe41a06b0 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.618+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdfdc005020 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.618+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdfdc0116d0 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.618+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdfe41a6e10 con 0x7fdfe4108cf0 2024-09-21T02:58:40.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.619+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdfe4102e70 con 0x7fdfe4108cf0 2024-09-21T02:58:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.621+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 20) ==== 154529+0+0 (secure 0 0 0) 0x7fdfdc0051d0 con 0x7fdfe4108cf0 2024-09-21T02:58:40.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.622+0000 7fdfd37fe640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 0x7fdfcc0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.622+0000 7fdfe259c640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 0x7fdfcc0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.622+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fdfdc0e7840 con 0x7fdfe4108cf0 2024-09-21T02:58:40.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.623+0000 7fdfe259c640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 0x7fdfcc0b8af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4031440 tx=0x7fdfd4032ed0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:40.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.626+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdfdc016030 con 0x7fdfe4108cf0 2024-09-21T02:58:40.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.844+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdfe4063800 con 0x7fdfcc0b65e0 2024-09-21T02:58:40.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.871+0000 7fdfd37fe640 1 -- 172.21.15.16:0/4031822677 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdfe4063800 con 0x7fdfcc0b65e0 2024-09-21T02:58:40.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:40 smithi016 ceph-mon[31253]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-21T02:58:40.873 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:40 smithi016 ceph-mon[31253]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 msgr2=0x7fdfcc0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:40.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 0x7fdfcc0b8af0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdfd4031440 tx=0x7fdfd4032ed0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 msgr2=0x7fdfe419fc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fdfdc00eaa0 tx=0x7fdfdc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 shutdown_connections 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdfcc0b65e0 0x7fdfcc0b8af0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdfe4108cf0 0x7fdfe419fc90 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdfe41083a0 0x7fdfe419a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 --2- 172.21.15.16:0/4031822677 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdfe4102a60 0x7fdfe419a450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 >> 172.21.15.16:0/4031822677 conn(0x7fdfe4079730 msgr2=0x7fdfe4101430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 shutdown_connections 2024-09-21T02:58:40.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:40.874+0000 7fdfe8928640 1 -- 172.21.15.16:0/4031822677 wait complete. 2024-09-21T02:58:40.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:40 smithi123 ceph-mon[34645]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-21T02:58:40.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:40 smithi123 ceph-mon[34645]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:40 smithi018 ceph-mon[34587]: pgmap v266: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s wr, 4 op/s 2024-09-21T02:58:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:40 smithi018 ceph-mon[34587]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:41.589 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-09-21T02:58:41.860 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:41.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:41 smithi123 ceph-mon[34645]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:41.981 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:41 smithi016 ceph-mon[31253]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:42.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:41 smithi018 ceph-mon[34587]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2872405401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c102a80 msgr2=0x7fbf4c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2872405401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c102a80 0x7fbf4c102e90 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fbf48009900 tx=0x7fbf4802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2872405401 shutdown_connections 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2872405401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c108d10 0x7fbf4c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2872405401 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2872405401 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c102a80 0x7fbf4c102e90 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2872405401 >> 172.21.15.16:0/2872405401 conn(0x7fbf4c079740 msgr2=0x7fbf4c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2872405401 shutdown_connections 2024-09-21T02:58:42.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.404+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2872405401 wait complete. 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 Processor -- start 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 -- start start 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c102a80 0x7fbf4c10ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c108d10 0x7fbf4c110a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:42.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c113aa0 con 0x7fbf4c1083c0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c113920 con 0x7fbf4c108d10 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf53f4a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fbf4c113c20 con 0x7fbf4c102a80 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf514e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf514e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60686/0 (socket says 172.21.15.16:60686) 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf514e5640 1 -- 172.21.15.16:0/2498122173 learned_addr learned my addr 172.21.15.16:0/2498122173 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.405+0000 7fbf51ce6640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c102a80 0x7fbf4c10ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf524e7640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c108d10 0x7fbf4c110a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 -- 172.21.15.16:0/2498122173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c102a80 msgr2=0x7fbf4c10ffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c102a80 0x7fbf4c10ffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 -- 172.21.15.16:0/2498122173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c108d10 msgr2=0x7fbf4c110a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c108d10 0x7fbf4c110a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 -- 172.21.15.16:0/2498122173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf4c1a8700 con 0x7fbf4c1083c0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf514e5640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c970 tx=0x7fbf3c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c014070 con 0x7fbf4c1083c0 2024-09-21T02:58:42.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf3c004440 con 0x7fbf4c1083c0 2024-09-21T02:58:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf3c003e50 con 0x7fbf4c1083c0 2024-09-21T02:58:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf4c1a89f0 con 0x7fbf4c1083c0 2024-09-21T02:58:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.406+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf4c1a8ee0 con 0x7fbf4c1083c0 2024-09-21T02:58:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.408+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf4c103d00 con 0x7fbf4c1083c0 2024-09-21T02:58:42.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.409+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fbf3c017070 con 0x7fbf4c1083c0 2024-09-21T02:58:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.410+0000 7fbf42ffd640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 0x7fbf380b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.410+0000 7fbf51ce6640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 0x7fbf380b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:42.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.410+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fbf3c0e7710 con 0x7fbf4c1083c0 2024-09-21T02:58:42.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.410+0000 7fbf51ce6640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 0x7fbf380b8ca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:42.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.413+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fbf3c01e020 con 0x7fbf4c1083c0 2024-09-21T02:58:42.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.631+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fbf4c063800 con 0x7fbf380b6790 2024-09-21T02:58:42.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.638+0000 7fbf42ffd640 1 -- 172.21.15.16:0/2498122173 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fbf4c063800 con 0x7fbf380b6790 2024-09-21T02:58:42.640 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_0/b711c444-36a2-4306-ac37-c135224ef88c 2024-09-21T02:58:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 msgr2=0x7fbf380b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 0x7fbf380b8ca0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fbf48000c00 tx=0x7fbf48031400 comp rx=0 tx=0).stop 2024-09-21T02:58:42.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 msgr2=0x7fbf4c110510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fbf3c00c970 tx=0x7fbf3c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 shutdown_connections 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fbf380b6790 0x7fbf380b8ca0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fbf4c1083c0 0x7fbf4c110510 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fbf4c108d10 0x7fbf4c110a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 --2- 172.21.15.16:0/2498122173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbf4c102a80 0x7fbf4c10ffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.640+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 >> 172.21.15.16:0/2498122173 conn(0x7fbf4c079740 msgr2=0x7fbf4c101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.641+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 shutdown_connections 2024-09-21T02:58:42.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:42.641+0000 7fbf53f4a640 1 -- 172.21.15.16:0/2498122173 wait complete. 2024-09-21T02:58:42.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:42 smithi016 ceph-mon[31253]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-21T02:58:42.937 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:42 smithi016 ceph-mon[31253]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T02:58:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:42 smithi123 ceph-mon[34645]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-21T02:58:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:42 smithi123 ceph-mon[34645]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T02:58:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:42 smithi018 ceph-mon[34587]: pgmap v267: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1.2 KiB/s wr, 5 op/s 2024-09-21T02:58:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:42 smithi018 ceph-mon[34587]: mgrmap e21: x(active, since 6m), standbys: z, y 2024-09-21T02:58:43.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs subvolume create cephfs sv_1 --group_name qa 2024-09-21T02:58:43.544 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:43.910 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:43 smithi016 ceph-mon[31253]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:43.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:43 smithi123 ceph-mon[34645]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:44.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 -- 172.21.15.16:0/915095951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8102ae0 msgr2=0x7fdec8102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:44.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 --2- 172.21.15.16:0/915095951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8102ae0 0x7fdec8102ef0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8009900 tx=0x7fdeb802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:44.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 -- 172.21.15.16:0/915095951 shutdown_connections 2024-09-21T02:58:44.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 --2- 172.21.15.16:0/915095951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 0x7fdec810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 --2- 172.21.15.16:0/915095951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec81083d0 0x7fdec81087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 --2- 172.21.15.16:0/915095951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec8102ae0 0x7fdec8102ef0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 -- 172.21.15.16:0/915095951 >> 172.21.15.16:0/915095951 conn(0x7fdec8079760 msgr2=0x7fdec8079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:44.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 -- 172.21.15.16:0/915095951 shutdown_connections 2024-09-21T02:58:44.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.006+0000 7fdecd264640 1 -- 172.21.15.16:0/915095951 wait complete. 2024-09-21T02:58:44.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 Processor -- start 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 -- start start 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 0x7fdec8104280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 0x7fdec8104d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8112990 con 0x7fdec81083d0 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8112810 con 0x7fdec8102ae0 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdecd264640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8112b10 con 0x7fdec8108d20 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdebffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdebffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60694/0 (socket says 172.21.15.16:60694) 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdebffff640 1 -- 172.21.15.16:0/250608614 learned_addr learned my addr 172.21.15.16:0/250608614 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdec77fe640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 0x7fdec8104d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.007+0000 7fdec6ffd640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 0x7fdec8104280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 -- 172.21.15.16:0/250608614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 msgr2=0x7fdec8104d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:44.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 0x7fdec8104d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 -- 172.21.15.16:0/250608614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 msgr2=0x7fdec8104280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 0x7fdec8104280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 -- 172.21.15.16:0/250608614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec81a86c0 con 0x7fdec81083d0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdec6ffd640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 0x7fdec8104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdebffff640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0014070 con 0x7fdec81083d0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdeb0004440 con 0x7fdec81083d0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec81a89b0 con 0x7fdec81083d0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdeb0003e50 con 0x7fdec81083d0 2024-09-21T02:58:44.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.008+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec81a8ef0 con 0x7fdec81083d0 2024-09-21T02:58:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.009+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec8103d10 con 0x7fdec81083d0 2024-09-21T02:58:44.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.011+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fdeb0017070 con 0x7fdec81083d0 2024-09-21T02:58:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.012+0000 7fdec4ff9640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 0x7fdeb40b8ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.012+0000 7fdec6ffd640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 0x7fdeb40b8ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.012+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fdeb00e77f0 con 0x7fdec81083d0 2024-09-21T02:58:44.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.013+0000 7fdec6ffd640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 0x7fdeb40b8ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000c00 tx=0x7fdeb8009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:44.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.015+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdeb001e030 con 0x7fdec81083d0 2024-09-21T02:58:44.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:43 smithi018 ceph-mon[34587]: from='client.14946 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:44.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.232+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fdec806bbd0 con 0x7fdeb40b6790 2024-09-21T02:58:44.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.261+0000 7fdec4ff9640 1 -- 172.21.15.16:0/250608614 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fdec806bbd0 con 0x7fdeb40b6790 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 msgr2=0x7fdeb40b8ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 0x7fdeb40b8ca0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8000c00 tx=0x7fdeb8009680 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 msgr2=0x7fdec81047c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7fdeb000c970 tx=0x7fdeb000ce30 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 shutdown_connections 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdeb40b6790 0x7fdeb40b8ca0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdec81083d0 0x7fdec81047c0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdec8108d20 0x7fdec8104d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 --2- 172.21.15.16:0/250608614 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdec8102ae0 0x7fdec8104280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 >> 172.21.15.16:0/250608614 conn(0x7fdec8079760 msgr2=0x7fdec81019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 shutdown_connections 2024-09-21T02:58:44.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:44.263+0000 7fdecd264640 1 -- 172.21.15.16:0/250608614 wait complete. 2024-09-21T02:58:44.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:44 smithi016 ceph-mon[31253]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T02:58:44.921 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-09-21T02:58:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:44 smithi123 ceph-mon[34645]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T02:58:45.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:44 smithi018 ceph-mon[34587]: pgmap v268: 97 pgs: 97 active+clean; 579 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 -- 172.21.15.16:0/2917045363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8108cf0 msgr2=0x7ff8c810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 --2- 172.21.15.16:0/2917045363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8108cf0 0x7ff8c810f520 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7ff8c0009900 tx=0x7ff8c002a410 comp rx=0 tx=0).stop 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 -- 172.21.15.16:0/2917045363 shutdown_connections 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 --2- 172.21.15.16:0/2917045363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8108cf0 0x7ff8c810f520 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 --2- 172.21.15.16:0/2917045363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c81083a0 0x7ff8c81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 --2- 172.21.15.16:0/2917045363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c8102a60 0x7ff8c8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 -- 172.21.15.16:0/2917045363 >> 172.21.15.16:0/2917045363 conn(0x7ff8c8079730 msgr2=0x7ff8c8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 -- 172.21.15.16:0/2917045363 shutdown_connections 2024-09-21T02:58:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.656+0000 7ff8cd691640 1 -- 172.21.15.16:0/2917045363 wait complete. 2024-09-21T02:58:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 Processor -- start 2024-09-21T02:58:45.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 -- start start 2024-09-21T02:58:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 0x7ff8c819a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c8108cf0 0x7ff8c819fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:45.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8112960 con 0x7ff8c8102a60 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c81127e0 con 0x7ff8c8108cf0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8cd691640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff8c8112ae0 con 0x7ff8c81083a0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8c6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46334/0 (socket says 172.21.15.16:46334) 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8c6ffd640 1 -- 172.21.15.16:0/1789950656 learned_addr learned my addr 172.21.15.16:0/1789950656 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.657+0000 7ff8c77fe640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c8108cf0 0x7ff8c819fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c67fc640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 0x7ff8c819a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 -- 172.21.15.16:0/1789950656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 msgr2=0x7ff8c819a980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 0x7ff8c819a980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 -- 172.21.15.16:0/1789950656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c8108cf0 msgr2=0x7ff8c819fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c8108cf0 0x7ff8c819fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 -- 172.21.15.16:0/1789950656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8c81a0350 con 0x7ff8c8102a60 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c67fc640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 0x7ff8c819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8c6ffd640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800ea80 tx=0x7ff8b800ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:45.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b800ce40 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8c81a0640 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8b8005020 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8b80116d0 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.658+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8c81a6da0 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.659+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8c8102e70 con 0x7ff8c8102a60 2024-09-21T02:58:45.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.661+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff8b80054d0 con 0x7ff8c8102a60 2024-09-21T02:58:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.662+0000 7ff8bffff640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 0x7ff8b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.662+0000 7ff8c67fc640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 0x7ff8b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:45.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.662+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7ff8b80e78b0 con 0x7ff8c8102a60 2024-09-21T02:58:45.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.662+0000 7ff8c67fc640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 0x7ff8b40b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806a2a0 tx=0x7ff8b0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:45.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.665+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff8b8016030 con 0x7ff8c8102a60 2024-09-21T02:58:45.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.883+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7ff8c8063800 con 0x7ff8b40b65f0 2024-09-21T02:58:45.884 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:45 smithi016 ceph-mon[31253]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:45.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.889+0000 7ff8bffff640 1 -- 172.21.15.16:0/1789950656 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7ff8c8063800 con 0x7ff8b40b65f0 2024-09-21T02:58:45.890 INFO:teuthology.orchestra.run.smithi016.stdout:/volumes/qa/sv_1/1bf5d314-c769-4152-adce-e93b2c8ff91a 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 msgr2=0x7ff8b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 0x7ff8b40b8b00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff8c806a2a0 tx=0x7ff8b0005f90 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 msgr2=0x7ff8c819a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7ff8b800ea80 tx=0x7ff8b800ef40 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 shutdown_connections 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff8b40b65f0 0x7ff8b40b8b00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff8c8108cf0 0x7ff8c819fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff8c81083a0 0x7ff8c819a980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 --2- 172.21.15.16:0/1789950656 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff8c8102a60 0x7ff8c819a3e0 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 >> 172.21.15.16:0/1789950656 conn(0x7ff8c8079730 msgr2=0x7ff8c806b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 shutdown_connections 2024-09-21T02:58:45.893 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:45.891+0000 7ff8cd691640 1 -- 172.21.15.16:0/1789950656 wait complete. 2024-09-21T02:58:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:45 smithi018 ceph-mon[34587]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:45 smithi123 ceph-mon[34645]: from='client.14952 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:46.547 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:58:46.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:46 smithi016 ceph-mon[31253]: pgmap v269: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:46.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:46 smithi016 ceph-mon[31253]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:46.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:46 smithi123 ceph-mon[34645]: pgmap v269: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:46.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:46 smithi123 ceph-mon[34645]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:46 smithi018 ceph-mon[34587]: pgmap v269: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:46 smithi018 ceph-mon[34587]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:58:47.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 -- 172.21.15.16:0/66570558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec109df0 msgr2=0x7feaec110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 --2- 172.21.15.16:0/66570558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec109df0 0x7feaec110620 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7feae8009ca0 tx=0x7feae802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 -- 172.21.15.16:0/66570558 shutdown_connections 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 --2- 172.21.15.16:0/66570558 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec109df0 0x7feaec110620 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 --2- 172.21.15.16:0/66570558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec1094a0 0x7feaec1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 --2- 172.21.15.16:0/66570558 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec103b60 0x7feaec103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 -- 172.21.15.16:0/66570558 >> 172.21.15.16:0/66570558 conn(0x7feaec07b850 msgr2=0x7feaec07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 -- 172.21.15.16:0/66570558 shutdown_connections 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.217+0000 7feaf42e5640 1 -- 172.21.15.16:0/66570558 wait complete. 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 Processor -- start 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 -- start start 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec103b60 0x7feaec19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec109df0 0x7feaec19fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:47.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feaec113ab0 con 0x7feaec1094a0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feaec113930 con 0x7feaec109df0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.218+0000 7feaf42e5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feaec113c30 con 0x7feaec103b60 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46356/0 (socket says 172.21.15.16:46356) 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 -- 172.21.15.16:0/746934839 learned_addr learned my addr 172.21.15.16:0/746934839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf2081640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec103b60 0x7feaec19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf2882640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec109df0 0x7feaec19fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 -- 172.21.15.16:0/746934839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec103b60 msgr2=0x7feaec19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec103b60 0x7feaec19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 -- 172.21.15.16:0/746934839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec109df0 msgr2=0x7feaec19fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec109df0 0x7feaec19fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 -- 172.21.15.16:0/746934839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feaec1a02f0 con 0x7feaec1094a0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf1880640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c8c0 tx=0x7feadc00cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc007bb0 con 0x7feaec1094a0 2024-09-21T02:58:47.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feaec1a05e0 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feadc007d50 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feadc0187c0 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.219+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feaec1a6e00 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.220+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feaec103f70 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.223+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7feadc00fa20 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.224+0000 7feae37fe640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 0x7fead80b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.224+0000 7feaf2081640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 0x7fead80b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.224+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7feadc0e7850 con 0x7feaec1094a0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.224+0000 7feaf2081640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 0x7fead80b8b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feae400bbe0 tx=0x7feae400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:47.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.227+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7feadc014030 con 0x7feaec1094a0 2024-09-21T02:58:47.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.454+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7feaec19b570 con 0x7feaec1094a0 2024-09-21T02:58:47.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.455+0000 7feae37fe640 1 -- 172.21.15.16:0/746934839 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7feaec19b570 con 0x7feaec1094a0 2024-09-21T02:58:47.456 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:47.457 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:58:47.457 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 msgr2=0x7fead80b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 0x7fead80b8b30 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7feae400bbe0 tx=0x7feae400b390 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 msgr2=0x7feaec19a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7feadc00c8c0 tx=0x7feadc00cd80 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 shutdown_connections 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fead80b6620 0x7fead80b8b30 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feaec1094a0 0x7feaec19a9e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feaec109df0 0x7feaec19fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 --2- 172.21.15.16:0/746934839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaec103b60 0x7feaec19a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 >> 172.21.15.16:0/746934839 conn(0x7feaec07b850 msgr2=0x7feaec105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 shutdown_connections 2024-09-21T02:58:47.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:47.457+0000 7feaf42e5640 1 -- 172.21.15.16:0/746934839 wait complete. 2024-09-21T02:58:47.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:47 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/746934839' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:47.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:47 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/746934839' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:48.060 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:58:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:47 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/746934839' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:58:48.316 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:48.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- 172.21.15.16:0/1306552684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 msgr2=0x7ff2f0110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/1306552684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f0110330 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8009ca0 tx=0x7ff2e802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- 172.21.15.16:0/1306552684 shutdown_connections 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/1306552684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f0110330 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/1306552684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f0105330 0x7ff2f010d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/1306552684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f01049e0 0x7ff2f0104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- 172.21.15.16:0/1306552684 >> 172.21.15.16:0/1306552684 conn(0x7ff2f0079750 msgr2=0x7ff2f0079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- 172.21.15.16:0/1306552684 shutdown_connections 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- 172.21.15.16:0/1306552684 wait complete. 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 Processor -- start 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.760+0000 7ff2f71ef640 1 -- start start 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01049e0 0x7ff2f019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0105330 0x7ff2f019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:48.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0113aa0 con 0x7ff2f010de40 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0113920 con 0x7ff2f0105330 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f71ef640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2f0113c20 con 0x7ff2f01049e0 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46372/0 (socket says 172.21.15.16:46372) 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 -- 172.21.15.16:0/2582461222 learned_addr learned my addr 172.21.15.16:0/2582461222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2e7fff640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0105330 0x7ff2f019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f4f8b640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01049e0 0x7ff2f019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 -- 172.21.15.16:0/2582461222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01049e0 msgr2=0x7ff2f019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01049e0 0x7ff2f019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 -- 172.21.15.16:0/2582461222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0105330 msgr2=0x7ff2f019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:48.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0105330 0x7ff2f019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 -- 172.21.15.16:0/2582461222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2f01a0380 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2f578c640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005cc0 tx=0x7ff2e802f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.761+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e8003bf0 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.762+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2e8003d90 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.762+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2f01a0610 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.762+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2e802fbc0 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.762+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2f01a6dd0 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.763+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2f006c280 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.765+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff2e802fd60 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.765+0000 7ff2e5ffb640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 0x7ff2d80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.765+0000 7ff2f4f8b640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 0x7ff2d80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.765+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7ff2e8108a80 con 0x7ff2f010de40 2024-09-21T02:58:48.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.766+0000 7ff2f4f8b640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 0x7ff2d80b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006c980 tx=0x7ff2ec00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:48.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.769+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2e80cf1d0 con 0x7ff2f010de40 2024-09-21T02:58:48.958 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.957+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff2f0104df0 con 0x7ff2f010de40 2024-09-21T02:58:48.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.958+0000 7ff2e5ffb640 1 -- 172.21.15.16:0/2582461222 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) ==== 74+0+22582 (secure 0 0 0) 0x7ff2e80d35b0 con 0x7ff2f010de40 2024-09-21T02:58:48.960 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:48.960 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":75,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:28.365049+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:58:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 msgr2=0x7ff2d80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 0x7ff2d80b8b80 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff2f006c980 tx=0x7ff2ec00b390 comp rx=0 tx=0).stop 2024-09-21T02:58:48.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 msgr2=0x7ff2f019fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7ff2e8005cc0 tx=0x7ff2e802f380 comp rx=0 tx=0).stop 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 shutdown_connections 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2d80b6670 0x7ff2d80b8b80 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2f010de40 0x7ff2f019fc50 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2f0105330 0x7ff2f019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 --2- 172.21.15.16:0/2582461222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2f01049e0 0x7ff2f019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 >> 172.21.15.16:0/2582461222 conn(0x7ff2f0079750 msgr2=0x7ff2f0105820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 shutdown_connections 2024-09-21T02:58:48.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:48.961+0000 7ff2f71ef640 1 -- 172.21.15.16:0/2582461222 wait complete. 2024-09-21T02:58:48.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:48 smithi123 ceph-mon[34645]: pgmap v270: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:48.973 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:48 smithi016 ceph-mon[31253]: pgmap v270: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:48 smithi018 ceph-mon[34587]: pgmap v270: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:49.640 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:58:49.640 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:58:49.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:49 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2582461222' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:49.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:49.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:49 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2582461222' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:50.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:49 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2582461222' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 -- 172.21.15.16:0/3362460895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 msgr2=0x7fc80c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 --2- 172.21.15.16:0/3362460895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c1087b0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc800009ca0 tx=0x7fc80002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 -- 172.21.15.16:0/3362460895 shutdown_connections 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 --2- 172.21.15.16:0/3362460895 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c1087b0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 --2- 172.21.15.16:0/3362460895 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 0x7fc80c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 --2- 172.21.15.16:0/3362460895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 0x7fc80c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 -- 172.21.15.16:0/3362460895 >> 172.21.15.16:0/3362460895 conn(0x7fc80c079730 msgr2=0x7fc80c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 -- 172.21.15.16:0/3362460895 shutdown_connections 2024-09-21T02:58:50.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.400+0000 7fc811095640 1 -- 172.21.15.16:0/3362460895 wait complete. 2024-09-21T02:58:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 Processor -- start 2024-09-21T02:58:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 -- start start 2024-09-21T02:58:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 0x7fc80c19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:50.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 0x7fc80c19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c112960 con 0x7fc80c1083a0 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c1127e0 con 0x7fc80c108cf0 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.401+0000 7fc811095640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc80c112ae0 con 0x7fc80c102a60 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46380/0 (socket says 172.21.15.16:46380) 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 -- 172.21.15.16:0/3008716633 learned_addr learned my addr 172.21.15.16:0/3008716633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80ad9d640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 0x7fc80c19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80b59e640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 0x7fc80c19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 -- 172.21.15.16:0/3008716633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 msgr2=0x7fc80c19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 0x7fc80c19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 -- 172.21.15.16:0/3008716633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 msgr2=0x7fc80c19bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 0x7fc80c19bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 -- 172.21.15.16:0/3008716633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc80c1a6520 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc80a59c640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc8000319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc800003bf0 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc80c1a67b0 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc800003d90 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.402+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc80002fb60 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.403+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc80c1a6d40 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.404+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc80c102e70 con 0x7fc80c1083a0 2024-09-21T02:58:50.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.405+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc80002fd00 con 0x7fc80c1083a0 2024-09-21T02:58:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.406+0000 7fc7f3fff640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 0x7fc7f40b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.406+0000 7fc80ad9d640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 0x7fc7f40b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:50.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.406+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7fc800108740 con 0x7fc80c1083a0 2024-09-21T02:58:50.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.406+0000 7fc80ad9d640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 0x7fc7f40b8b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80c06b9c0 tx=0x7fc7fc00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:50.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.410+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc8000cee30 con 0x7fc80c1083a0 2024-09-21T02:58:50.595 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.594+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fc80c10ab40 con 0x7fc7f40b6620 2024-09-21T02:58:50.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.599+0000 7fc7f3fff640 1 -- 172.21.15.16:0/3008716633 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258611 (secure 0 0 0) 0x7fc80c10ab40 con 0x7fc7f40b6620 2024-09-21T02:58:50.603 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:50.604 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 msgr2=0x7fc7f40b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 0x7fc7f40b8b30 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fc80c06b9c0 tx=0x7fc7fc00b410 comp rx=0 tx=0).stop 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 msgr2=0x7fc80c19b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fc800006010 tx=0x7fc8000319f0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 shutdown_connections 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc7f40b6620 0x7fc7f40b8b30 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc80c1083a0 0x7fc80c19b7d0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc80c108cf0 0x7fc80c19bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 --2- 172.21.15.16:0/3008716633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc80c102a60 0x7fc80c19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 >> 172.21.15.16:0/3008716633 conn(0x7fc80c079730 msgr2=0x7fc80c104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 shutdown_connections 2024-09-21T02:58:50.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:50.602+0000 7fc811095640 1 -- 172.21.15.16:0/3008716633 wait complete. 2024-09-21T02:58:50.897 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:50 smithi016 ceph-mon[31253]: pgmap v271: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:50 smithi123 ceph-mon[34645]: pgmap v271: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:50 smithi018 ceph-mon[34587]: pgmap v271: 97 pgs: 97 active+clean; 606 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 3.0 KiB/s wr, 3 op/s 2024-09-21T02:58:51.287 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":271,"stamp":"2024-09-21T02:58:49.147063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":620415,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":110,"num_read_kb":114,"num_write":186,"num_write_kb":2200,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":145,"ondisk_log_size":145,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385632,"kb_used_data":11380,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556192,"statfs":{"total":1151940427776,"available":1151545540608,"internally_reserved":0,"allocated":11653120,"data_stored":6644922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":27637,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":18,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003144"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657798+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:58:28.657798+0000","last_peered":"2024-09-21T02:58:28.657798+0000","last_clean":"2024-09-21T02:58:28.657798+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:58:28.657798+0000","last_undegraded":"2024-09-21T02:58:28.657798+0000","last_fullsized":"2024-09-21T02:58:28.657798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653242+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:58:28.653242+0000","last_peered":"2024-09-21T02:58:28.653242+0000","last_clean":"2024-09-21T02:58:28.653242+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:58:28.653242+0000","last_undegraded":"2024-09-21T02:58:28.653242+0000","last_fullsized":"2024-09-21T02:58:28.653242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652869+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:58:28.652869+0000","last_peered":"2024-09-21T02:58:28.652869+0000","last_clean":"2024-09-21T02:58:28.652869+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:58:28.652869+0000","last_undegraded":"2024-09-21T02:58:28.652869+0000","last_fullsized":"2024-09-21T02:58:28.652869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655311+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:58:28.655311+0000","last_peered":"2024-09-21T02:58:28.655311+0000","last_clean":"2024-09-21T02:58:28.655311+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:58:28.655311+0000","last_undegraded":"2024-09-21T02:58:28.655311+0000","last_fullsized":"2024-09-21T02:58:28.655311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651117+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:58:28.651117+0000","last_peered":"2024-09-21T02:58:28.651117+0000","last_clean":"2024-09-21T02:58:28.651117+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:58:28.651117+0000","last_undegraded":"2024-09-21T02:58:28.651117+0000","last_fullsized":"2024-09-21T02:58:28.651117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648621+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:58:28.648621+0000","last_peered":"2024-09-21T02:58:28.648621+0000","last_clean":"2024-09-21T02:58:28.648621+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:58:28.648621+0000","last_undegraded":"2024-09-21T02:58:28.648621+0000","last_fullsized":"2024-09-21T02:58:28.648621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640584+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:58:28.640584+0000","last_peered":"2024-09-21T02:58:28.640584+0000","last_clean":"2024-09-21T02:58:28.640584+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:58:28.640584+0000","last_undegraded":"2024-09-21T02:58:28.640584+0000","last_fullsized":"2024-09-21T02:58:28.640584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648114+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:58:28.648114+0000","last_peered":"2024-09-21T02:58:28.648114+0000","last_clean":"2024-09-21T02:58:28.648114+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:58:28.648114+0000","last_undegraded":"2024-09-21T02:58:28.648114+0000","last_fullsized":"2024-09-21T02:58:28.648114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653175+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:58:28.653175+0000","last_peered":"2024-09-21T02:58:28.653175+0000","last_clean":"2024-09-21T02:58:28.653175+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:58:28.653175+0000","last_undegraded":"2024-09-21T02:58:28.653175+0000","last_fullsized":"2024-09-21T02:58:28.653175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649403+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:58:28.649403+0000","last_peered":"2024-09-21T02:58:28.649403+0000","last_clean":"2024-09-21T02:58:28.649403+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:58:28.649403+0000","last_undegraded":"2024-09-21T02:58:28.649403+0000","last_fullsized":"2024-09-21T02:58:28.649403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641259+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:58:28.641259+0000","last_peered":"2024-09-21T02:58:28.641259+0000","last_clean":"2024-09-21T02:58:28.641259+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:58:28.641259+0000","last_undegraded":"2024-09-21T02:58:28.641259+0000","last_fullsized":"2024-09-21T02:58:28.641259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.654740+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:58:28.654740+0000","last_peered":"2024-09-21T02:58:28.654740+0000","last_clean":"2024-09-21T02:58:28.654740+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:58:28.654740+0000","last_undegraded":"2024-09-21T02:58:28.654740+0000","last_fullsized":"2024-09-21T02:58:28.654740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639835+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:58:28.639835+0000","last_peered":"2024-09-21T02:58:28.639835+0000","last_clean":"2024-09-21T02:58:28.639835+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:58:28.639835+0000","last_undegraded":"2024-09-21T02:58:28.639835+0000","last_fullsized":"2024-09-21T02:58:28.639835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652409+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:58:28.652409+0000","last_peered":"2024-09-21T02:58:28.652409+0000","last_clean":"2024-09-21T02:58:28.652409+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:58:28.652409+0000","last_undegraded":"2024-09-21T02:58:28.652409+0000","last_fullsized":"2024-09-21T02:58:28.652409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652048+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:58:28.652048+0000","last_peered":"2024-09-21T02:58:28.652048+0000","last_clean":"2024-09-21T02:58:28.652048+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:58:28.652048+0000","last_undegraded":"2024-09-21T02:58:28.652048+0000","last_fullsized":"2024-09-21T02:58:28.652048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652474+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:58:28.652474+0000","last_peered":"2024-09-21T02:58:28.652474+0000","last_clean":"2024-09-21T02:58:28.652474+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:58:28.652474+0000","last_undegraded":"2024-09-21T02:58:28.652474+0000","last_fullsized":"2024-09-21T02:58:28.652474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657879+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:58:28.657879+0000","last_peered":"2024-09-21T02:58:28.657879+0000","last_clean":"2024-09-21T02:58:28.657879+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:58:28.657879+0000","last_undegraded":"2024-09-21T02:58:28.657879+0000","last_fullsized":"2024-09-21T02:58:28.657879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651598+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:58:28.651598+0000","last_peered":"2024-09-21T02:58:28.651598+0000","last_clean":"2024-09-21T02:58:28.651598+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:58:28.651598+0000","last_undegraded":"2024-09-21T02:58:28.651598+0000","last_fullsized":"2024-09-21T02:58:28.651598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640396+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:58:28.640396+0000","last_peered":"2024-09-21T02:58:28.640396+0000","last_clean":"2024-09-21T02:58:28.640396+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:58:28.640396+0000","last_undegraded":"2024-09-21T02:58:28.640396+0000","last_fullsized":"2024-09-21T02:58:28.640396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657177+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:58:28.657177+0000","last_peered":"2024-09-21T02:58:28.657177+0000","last_clean":"2024-09-21T02:58:28.657177+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:58:28.657177+0000","last_undegraded":"2024-09-21T02:58:28.657177+0000","last_fullsized":"2024-09-21T02:58:28.657177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639873+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:58:28.639873+0000","last_peered":"2024-09-21T02:58:28.639873+0000","last_clean":"2024-09-21T02:58:28.639873+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:58:28.639873+0000","last_undegraded":"2024-09-21T02:58:28.639873+0000","last_fullsized":"2024-09-21T02:58:28.639873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651969+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:58:28.651969+0000","last_peered":"2024-09-21T02:58:28.651969+0000","last_clean":"2024-09-21T02:58:28.651969+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:58:28.651969+0000","last_undegraded":"2024-09-21T02:58:28.651969+0000","last_fullsized":"2024-09-21T02:58:28.651969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648273+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:58:28.648273+0000","last_peered":"2024-09-21T02:58:28.648273+0000","last_clean":"2024-09-21T02:58:28.648273+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:58:28.648273+0000","last_undegraded":"2024-09-21T02:58:28.648273+0000","last_fullsized":"2024-09-21T02:58:28.648273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.654551+0000","last_change":"2024-09-21T02:58:27.647504+0000","last_active":"2024-09-21T02:58:28.654551+0000","last_peered":"2024-09-21T02:58:28.654551+0000","last_clean":"2024-09-21T02:58:28.654551+0000","last_became_active":"2024-09-21T02:58:27.647051+0000","last_became_peered":"2024-09-21T02:58:27.647051+0000","last_unstale":"2024-09-21T02:58:28.654551+0000","last_undegraded":"2024-09-21T02:58:28.654551+0000","last_fullsized":"2024-09-21T02:58:28.654551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:21.890708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649102+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:58:28.649102+0000","last_peered":"2024-09-21T02:58:28.649102+0000","last_clean":"2024-09-21T02:58:28.649102+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:58:28.649102+0000","last_undegraded":"2024-09-21T02:58:28.649102+0000","last_fullsized":"2024-09-21T02:58:28.649102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640473+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:58:28.640473+0000","last_peered":"2024-09-21T02:58:28.640473+0000","last_clean":"2024-09-21T02:58:28.640473+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:58:28.640473+0000","last_undegraded":"2024-09-21T02:58:28.640473+0000","last_fullsized":"2024-09-21T02:58:28.640473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657699+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:58:28.657699+0000","last_peered":"2024-09-21T02:58:28.657699+0000","last_clean":"2024-09-21T02:58:28.657699+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:58:28.657699+0000","last_undegraded":"2024-09-21T02:58:28.657699+0000","last_fullsized":"2024-09-21T02:58:28.657699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639385+0000","last_change":"2024-09-21T02:58:27.649283+0000","last_active":"2024-09-21T02:58:28.639385+0000","last_peered":"2024-09-21T02:58:28.639385+0000","last_clean":"2024-09-21T02:58:28.639385+0000","last_became_active":"2024-09-21T02:58:27.648919+0000","last_became_peered":"2024-09-21T02:58:27.648919+0000","last_unstale":"2024-09-21T02:58:28.639385+0000","last_undegraded":"2024-09-21T02:58:28.639385+0000","last_fullsized":"2024-09-21T02:58:28.639385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:57:27.291907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653438+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:58:28.653438+0000","last_peered":"2024-09-21T02:58:28.653438+0000","last_clean":"2024-09-21T02:58:28.653438+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:58:28.653438+0000","last_undegraded":"2024-09-21T02:58:28.653438+0000","last_fullsized":"2024-09-21T02:58:28.653438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640008+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:58:28.640008+0000","last_peered":"2024-09-21T02:58:28.640008+0000","last_clean":"2024-09-21T02:58:28.640008+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:58:28.640008+0000","last_undegraded":"2024-09-21T02:58:28.640008+0000","last_fullsized":"2024-09-21T02:58:28.640008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640021+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:58:28.640021+0000","last_peered":"2024-09-21T02:58:28.640021+0000","last_clean":"2024-09-21T02:58:28.640021+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:58:28.640021+0000","last_undegraded":"2024-09-21T02:58:28.640021+0000","last_fullsized":"2024-09-21T02:58:28.640021+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658161+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:58:28.658161+0000","last_peered":"2024-09-21T02:58:28.658161+0000","last_clean":"2024-09-21T02:58:28.658161+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:58:28.658161+0000","last_undegraded":"2024-09-21T02:58:28.658161+0000","last_fullsized":"2024-09-21T02:58:28.658161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651265+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:58:28.651265+0000","last_peered":"2024-09-21T02:58:28.651265+0000","last_clean":"2024-09-21T02:58:28.651265+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:58:28.651265+0000","last_undegraded":"2024-09-21T02:58:28.651265+0000","last_fullsized":"2024-09-21T02:58:28.651265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652418+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:58:28.652418+0000","last_peered":"2024-09-21T02:58:28.652418+0000","last_clean":"2024-09-21T02:58:28.652418+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:58:28.652418+0000","last_undegraded":"2024-09-21T02:58:28.652418+0000","last_fullsized":"2024-09-21T02:58:28.652418+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652160+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:58:28.652160+0000","last_peered":"2024-09-21T02:58:28.652160+0000","last_clean":"2024-09-21T02:58:28.652160+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:58:28.652160+0000","last_undegraded":"2024-09-21T02:58:28.652160+0000","last_fullsized":"2024-09-21T02:58:28.652160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655294+0000","last_change":"2024-09-21T02:58:25.638302+0000","last_active":"2024-09-21T02:58:28.655294+0000","last_peered":"2024-09-21T02:58:28.655294+0000","last_clean":"2024-09-21T02:58:28.655294+0000","last_became_active":"2024-09-21T02:58:25.637899+0000","last_became_peered":"2024-09-21T02:58:25.637899+0000","last_unstale":"2024-09-21T02:58:28.655294+0000","last_undegraded":"2024-09-21T02:58:28.655294+0000","last_fullsized":"2024-09-21T02:58:28.655294+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:57.896986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639186+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:58:28.639186+0000","last_peered":"2024-09-21T02:58:28.639186+0000","last_clean":"2024-09-21T02:58:28.639186+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:58:28.639186+0000","last_undegraded":"2024-09-21T02:58:28.639186+0000","last_fullsized":"2024-09-21T02:58:28.639186+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648498+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:58:28.648498+0000","last_peered":"2024-09-21T02:58:28.648498+0000","last_clean":"2024-09-21T02:58:28.648498+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:58:28.648498+0000","last_undegraded":"2024-09-21T02:58:28.648498+0000","last_fullsized":"2024-09-21T02:58:28.648498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653445+0000","last_change":"2024-09-21T02:58:25.640688+0000","last_active":"2024-09-21T02:58:28.653445+0000","last_peered":"2024-09-21T02:58:28.653445+0000","last_clean":"2024-09-21T02:58:28.653445+0000","last_became_active":"2024-09-21T02:58:25.640306+0000","last_became_peered":"2024-09-21T02:58:25.640306+0000","last_unstale":"2024-09-21T02:58:28.653445+0000","last_undegraded":"2024-09-21T02:58:28.653445+0000","last_fullsized":"2024-09-21T02:58:28.653445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:04.808100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651262+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:58:28.651262+0000","last_peered":"2024-09-21T02:58:28.651262+0000","last_clean":"2024-09-21T02:58:28.651262+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:58:28.651262+0000","last_undegraded":"2024-09-21T02:58:28.651262+0000","last_fullsized":"2024-09-21T02:58:28.651262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648932+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:58:28.648932+0000","last_peered":"2024-09-21T02:58:28.648932+0000","last_clean":"2024-09-21T02:58:28.648932+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:58:28.648932+0000","last_undegraded":"2024-09-21T02:58:28.648932+0000","last_fullsized":"2024-09-21T02:58:28.648932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653072+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:58:28.653072+0000","last_peered":"2024-09-21T02:58:28.653072+0000","last_clean":"2024-09-21T02:58:28.653072+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:58:28.653072+0000","last_undegraded":"2024-09-21T02:58:28.653072+0000","last_fullsized":"2024-09-21T02:58:28.653072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640941+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:58:28.640941+0000","last_peered":"2024-09-21T02:58:28.640941+0000","last_clean":"2024-09-21T02:58:28.640941+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:58:28.640941+0000","last_undegraded":"2024-09-21T02:58:28.640941+0000","last_fullsized":"2024-09-21T02:58:28.640941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652508+0000","last_change":"2024-09-21T02:58:27.655900+0000","last_active":"2024-09-21T02:58:28.652508+0000","last_peered":"2024-09-21T02:58:28.652508+0000","last_clean":"2024-09-21T02:58:28.652508+0000","last_became_active":"2024-09-21T02:58:27.655522+0000","last_became_peered":"2024-09-21T02:58:27.655522+0000","last_unstale":"2024-09-21T02:58:28.652508+0000","last_undegraded":"2024-09-21T02:58:28.652508+0000","last_fullsized":"2024-09-21T02:58:28.652508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:58.361260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659590+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:58:28.659590+0000","last_peered":"2024-09-21T02:58:28.659590+0000","last_clean":"2024-09-21T02:58:28.659590+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:58:28.659590+0000","last_undegraded":"2024-09-21T02:58:28.659590+0000","last_fullsized":"2024-09-21T02:58:28.659590+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653794+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:58:28.653794+0000","last_peered":"2024-09-21T02:58:28.653794+0000","last_clean":"2024-09-21T02:58:28.653794+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:58:28.653794+0000","last_undegraded":"2024-09-21T02:58:28.653794+0000","last_fullsized":"2024-09-21T02:58:28.653794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'14","reported_seq":56,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:43.695529+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T02:58:43.695529+0000","last_peered":"2024-09-21T02:58:43.695529+0000","last_clean":"2024-09-21T02:58:43.695529+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T02:58:43.695529+0000","last_undegraded":"2024-09-21T02:58:43.695529+0000","last_fullsized":"2024-09-21T02:58:43.695529+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":29015,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":23,"num_read_kb":39,"num_write":14,"num_write_kb":33,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651717+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:58:28.651717+0000","last_peered":"2024-09-21T02:58:28.651717+0000","last_clean":"2024-09-21T02:58:28.651717+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:58:28.651717+0000","last_undegraded":"2024-09-21T02:58:28.651717+0000","last_fullsized":"2024-09-21T02:58:28.651717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649247+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:58:28.649247+0000","last_peered":"2024-09-21T02:58:28.649247+0000","last_clean":"2024-09-21T02:58:28.649247+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:58:28.649247+0000","last_undegraded":"2024-09-21T02:58:28.649247+0000","last_fullsized":"2024-09-21T02:58:28.649247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639467+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:58:28.639467+0000","last_peered":"2024-09-21T02:58:28.639467+0000","last_clean":"2024-09-21T02:58:28.639467+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:58:28.639467+0000","last_undegraded":"2024-09-21T02:58:28.639467+0000","last_fullsized":"2024-09-21T02:58:28.639467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651077+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:58:28.651077+0000","last_peered":"2024-09-21T02:58:28.651077+0000","last_clean":"2024-09-21T02:58:28.651077+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:58:28.651077+0000","last_undegraded":"2024-09-21T02:58:28.651077+0000","last_fullsized":"2024-09-21T02:58:28.651077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641137+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:58:28.641137+0000","last_peered":"2024-09-21T02:58:28.641137+0000","last_clean":"2024-09-21T02:58:28.641137+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:58:28.641137+0000","last_undegraded":"2024-09-21T02:58:28.641137+0000","last_fullsized":"2024-09-21T02:58:28.641137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652627+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:58:28.652627+0000","last_peered":"2024-09-21T02:58:28.652627+0000","last_clean":"2024-09-21T02:58:28.652627+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:58:28.652627+0000","last_undegraded":"2024-09-21T02:58:28.652627+0000","last_fullsized":"2024-09-21T02:58:28.652627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640595+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:58:28.640595+0000","last_peered":"2024-09-21T02:58:28.640595+0000","last_clean":"2024-09-21T02:58:28.640595+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:58:28.640595+0000","last_undegraded":"2024-09-21T02:58:28.640595+0000","last_fullsized":"2024-09-21T02:58:28.640595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659030+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:58:28.659030+0000","last_peered":"2024-09-21T02:58:28.659030+0000","last_clean":"2024-09-21T02:58:28.659030+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:58:28.659030+0000","last_undegraded":"2024-09-21T02:58:28.659030+0000","last_fullsized":"2024-09-21T02:58:28.659030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651476+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:58:28.651476+0000","last_peered":"2024-09-21T02:58:28.651476+0000","last_clean":"2024-09-21T02:58:28.651476+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:58:28.651476+0000","last_undegraded":"2024-09-21T02:58:28.651476+0000","last_fullsized":"2024-09-21T02:58:28.651476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658826+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:58:28.658826+0000","last_peered":"2024-09-21T02:58:28.658826+0000","last_clean":"2024-09-21T02:58:28.658826+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:58:28.658826+0000","last_undegraded":"2024-09-21T02:58:28.658826+0000","last_fullsized":"2024-09-21T02:58:28.658826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657319+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:58:28.657319+0000","last_peered":"2024-09-21T02:58:28.657319+0000","last_clean":"2024-09-21T02:58:28.657319+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:58:28.657319+0000","last_undegraded":"2024-09-21T02:58:28.657319+0000","last_fullsized":"2024-09-21T02:58:28.657319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.647905+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:58:28.647905+0000","last_peered":"2024-09-21T02:58:28.647905+0000","last_clean":"2024-09-21T02:58:28.647905+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:58:28.647905+0000","last_undegraded":"2024-09-21T02:58:28.647905+0000","last_fullsized":"2024-09-21T02:58:28.647905+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640947+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:58:28.640947+0000","last_peered":"2024-09-21T02:58:28.640947+0000","last_clean":"2024-09-21T02:58:28.640947+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:58:28.640947+0000","last_undegraded":"2024-09-21T02:58:28.640947+0000","last_fullsized":"2024-09-21T02:58:28.640947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659668+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:58:28.659668+0000","last_peered":"2024-09-21T02:58:28.659668+0000","last_clean":"2024-09-21T02:58:28.659668+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:58:28.659668+0000","last_undegraded":"2024-09-21T02:58:28.659668+0000","last_fullsized":"2024-09-21T02:58:28.659668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:29.459725+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:58:29.459725+0000","last_peered":"2024-09-21T02:58:29.459725+0000","last_clean":"2024-09-21T02:58:29.459725+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:58:29.459725+0000","last_undegraded":"2024-09-21T02:58:29.459725+0000","last_fullsized":"2024-09-21T02:58:29.459725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640370+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:58:28.640370+0000","last_peered":"2024-09-21T02:58:28.640370+0000","last_clean":"2024-09-21T02:58:28.640370+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:58:28.640370+0000","last_undegraded":"2024-09-21T02:58:28.640370+0000","last_fullsized":"2024-09-21T02:58:28.640370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641410+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:58:28.641410+0000","last_peered":"2024-09-21T02:58:28.641410+0000","last_clean":"2024-09-21T02:58:28.641410+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:58:28.641410+0000","last_undegraded":"2024-09-21T02:58:28.641410+0000","last_fullsized":"2024-09-21T02:58:28.641410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640685+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:58:28.640685+0000","last_peered":"2024-09-21T02:58:28.640685+0000","last_clean":"2024-09-21T02:58:28.640685+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:58:28.640685+0000","last_undegraded":"2024-09-21T02:58:28.640685+0000","last_fullsized":"2024-09-21T02:58:28.640685+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653863+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:58:28.653863+0000","last_peered":"2024-09-21T02:58:28.653863+0000","last_clean":"2024-09-21T02:58:28.653863+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:58:28.653863+0000","last_undegraded":"2024-09-21T02:58:28.653863+0000","last_fullsized":"2024-09-21T02:58:28.653863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648633+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:58:28.648633+0000","last_peered":"2024-09-21T02:58:28.648633+0000","last_clean":"2024-09-21T02:58:28.648633+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:58:28.648633+0000","last_undegraded":"2024-09-21T02:58:28.648633+0000","last_fullsized":"2024-09-21T02:58:28.648633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640683+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:58:28.640683+0000","last_peered":"2024-09-21T02:58:28.640683+0000","last_clean":"2024-09-21T02:58:28.640683+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:58:28.640683+0000","last_undegraded":"2024-09-21T02:58:28.640683+0000","last_fullsized":"2024-09-21T02:58:28.640683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639221+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:28.639221+0000","last_peered":"2024-09-21T02:58:28.639221+0000","last_clean":"2024-09-21T02:58:28.639221+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:58:28.639221+0000","last_undegraded":"2024-09-21T02:58:28.639221+0000","last_fullsized":"2024-09-21T02:58:28.639221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639702+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:58:28.639702+0000","last_peered":"2024-09-21T02:58:28.639702+0000","last_clean":"2024-09-21T02:58:28.639702+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:58:28.639702+0000","last_undegraded":"2024-09-21T02:58:28.639702+0000","last_fullsized":"2024-09-21T02:58:28.639702+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653188+0000","last_change":"2024-09-21T02:58:27.658246+0000","last_active":"2024-09-21T02:58:28.653188+0000","last_peered":"2024-09-21T02:58:28.653188+0000","last_clean":"2024-09-21T02:58:28.653188+0000","last_became_active":"2024-09-21T02:58:27.657143+0000","last_became_peered":"2024-09-21T02:58:27.657143+0000","last_unstale":"2024-09-21T02:58:28.653188+0000","last_undegraded":"2024-09-21T02:58:28.653188+0000","last_fullsized":"2024-09-21T02:58:28.653188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:56.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641359+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:58:28.641359+0000","last_peered":"2024-09-21T02:58:28.641359+0000","last_clean":"2024-09-21T02:58:28.641359+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:58:28.641359+0000","last_undegraded":"2024-09-21T02:58:28.641359+0000","last_fullsized":"2024-09-21T02:58:28.641359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649070+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:58:28.649070+0000","last_peered":"2024-09-21T02:58:28.649070+0000","last_clean":"2024-09-21T02:58:28.649070+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:58:28.649070+0000","last_undegraded":"2024-09-21T02:58:28.649070+0000","last_fullsized":"2024-09-21T02:58:28.649070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651315+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:58:28.651315+0000","last_peered":"2024-09-21T02:58:28.651315+0000","last_clean":"2024-09-21T02:58:28.651315+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:58:28.651315+0000","last_undegraded":"2024-09-21T02:58:28.651315+0000","last_fullsized":"2024-09-21T02:58:28.651315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:29.187908+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:58:29.187908+0000","last_peered":"2024-09-21T02:58:29.187908+0000","last_clean":"2024-09-21T02:58:29.187908+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:58:29.187908+0000","last_undegraded":"2024-09-21T02:58:29.187908+0000","last_fullsized":"2024-09-21T02:58:29.187908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651342+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:58:28.651342+0000","last_peered":"2024-09-21T02:58:28.651342+0000","last_clean":"2024-09-21T02:58:28.651342+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:58:28.651342+0000","last_undegraded":"2024-09-21T02:58:28.651342+0000","last_fullsized":"2024-09-21T02:58:28.651342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639623+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:58:28.639623+0000","last_peered":"2024-09-21T02:58:28.639623+0000","last_clean":"2024-09-21T02:58:28.639623+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:58:28.639623+0000","last_undegraded":"2024-09-21T02:58:28.639623+0000","last_fullsized":"2024-09-21T02:58:28.639623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'5","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:43.647072+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:58:43.647072+0000","last_peered":"2024-09-21T02:58:43.647072+0000","last_clean":"2024-09-21T02:58:43.647072+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:58:43.647072+0000","last_undegraded":"2024-09-21T02:58:43.647072+0000","last_fullsized":"2024-09-21T02:58:43.647072+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649252+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:58:28.649252+0000","last_peered":"2024-09-21T02:58:28.649252+0000","last_clean":"2024-09-21T02:58:28.649252+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:58:28.649252+0000","last_undegraded":"2024-09-21T02:58:28.649252+0000","last_fullsized":"2024-09-21T02:58:28.649252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.656897+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:58:28.656897+0000","last_peered":"2024-09-21T02:58:28.656897+0000","last_clean":"2024-09-21T02:58:28.656897+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:58:28.656897+0000","last_undegraded":"2024-09-21T02:58:28.656897+0000","last_fullsized":"2024-09-21T02:58:28.656897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653499+0000","last_change":"2024-09-21T02:58:27.656418+0000","last_active":"2024-09-21T02:58:28.653499+0000","last_peered":"2024-09-21T02:58:28.653499+0000","last_clean":"2024-09-21T02:58:28.653499+0000","last_became_active":"2024-09-21T02:58:27.656004+0000","last_became_peered":"2024-09-21T02:58:27.656004+0000","last_unstale":"2024-09-21T02:58:28.653499+0000","last_undegraded":"2024-09-21T02:58:28.653499+0000","last_fullsized":"2024-09-21T02:58:28.653499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:57.543580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652618+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:58:28.652618+0000","last_peered":"2024-09-21T02:58:28.652618+0000","last_clean":"2024-09-21T02:58:28.652618+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:58:28.652618+0000","last_undegraded":"2024-09-21T02:58:28.652618+0000","last_fullsized":"2024-09-21T02:58:28.652618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.650543+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:58:28.650543+0000","last_peered":"2024-09-21T02:58:28.650543+0000","last_clean":"2024-09-21T02:58:28.650543+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:58:28.650543+0000","last_undegraded":"2024-09-21T02:58:28.650543+0000","last_fullsized":"2024-09-21T02:58:28.650543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648456+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:58:28.648456+0000","last_peered":"2024-09-21T02:58:28.648456+0000","last_clean":"2024-09-21T02:58:28.648456+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:58:28.648456+0000","last_undegraded":"2024-09-21T02:58:28.648456+0000","last_fullsized":"2024-09-21T02:58:28.648456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658390+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:58:28.658390+0000","last_peered":"2024-09-21T02:58:28.658390+0000","last_clean":"2024-09-21T02:58:28.658390+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:58:28.658390+0000","last_undegraded":"2024-09-21T02:58:28.658390+0000","last_fullsized":"2024-09-21T02:58:28.658390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639964+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:58:28.639964+0000","last_peered":"2024-09-21T02:58:28.639964+0000","last_clean":"2024-09-21T02:58:28.639964+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:58:28.639964+0000","last_undegraded":"2024-09-21T02:58:28.639964+0000","last_fullsized":"2024-09-21T02:58:28.639964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648778+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:58:28.648778+0000","last_peered":"2024-09-21T02:58:28.648778+0000","last_clean":"2024-09-21T02:58:28.648778+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:58:28.648778+0000","last_undegraded":"2024-09-21T02:58:28.648778+0000","last_fullsized":"2024-09-21T02:58:28.648778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655405+0000","last_change":"2024-09-21T02:58:25.640895+0000","last_active":"2024-09-21T02:58:28.655405+0000","last_peered":"2024-09-21T02:58:28.655405+0000","last_clean":"2024-09-21T02:58:28.655405+0000","last_became_active":"2024-09-21T02:58:25.640488+0000","last_became_peered":"2024-09-21T02:58:25.640488+0000","last_unstale":"2024-09-21T02:58:28.655405+0000","last_undegraded":"2024-09-21T02:58:28.655405+0000","last_fullsized":"2024-09-21T02:58:28.655405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640922+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:58:28.640922+0000","last_peered":"2024-09-21T02:58:28.640922+0000","last_clean":"2024-09-21T02:58:28.640922+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:58:28.640922+0000","last_undegraded":"2024-09-21T02:58:28.640922+0000","last_fullsized":"2024-09-21T02:58:28.640922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653327+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:58:28.653327+0000","last_peered":"2024-09-21T02:58:28.653327+0000","last_clean":"2024-09-21T02:58:28.653327+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:58:28.653327+0000","last_undegraded":"2024-09-21T02:58:28.653327+0000","last_fullsized":"2024-09-21T02:58:28.653327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658913+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:58:28.658913+0000","last_peered":"2024-09-21T02:58:28.658913+0000","last_clean":"2024-09-21T02:58:28.658913+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:58:28.658913+0000","last_undegraded":"2024-09-21T02:58:28.658913+0000","last_fullsized":"2024-09-21T02:58:28.658913+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653465+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:58:28.653465+0000","last_peered":"2024-09-21T02:58:28.653465+0000","last_clean":"2024-09-21T02:58:28.653465+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:58:28.653465+0000","last_undegraded":"2024-09-21T02:58:28.653465+0000","last_fullsized":"2024-09-21T02:58:28.653465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655188+0000","last_change":"2024-09-21T02:58:25.642383+0000","last_active":"2024-09-21T02:58:28.655188+0000","last_peered":"2024-09-21T02:58:28.655188+0000","last_clean":"2024-09-21T02:58:28.655188+0000","last_became_active":"2024-09-21T02:58:25.641933+0000","last_became_peered":"2024-09-21T02:58:25.641933+0000","last_unstale":"2024-09-21T02:58:28.655188+0000","last_undegraded":"2024-09-21T02:58:28.655188+0000","last_fullsized":"2024-09-21T02:58:28.655188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:34.484094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652207+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:58:28.652207+0000","last_peered":"2024-09-21T02:58:28.652207+0000","last_clean":"2024-09-21T02:58:28.652207+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:58:28.652207+0000","last_undegraded":"2024-09-21T02:58:28.652207+0000","last_fullsized":"2024-09-21T02:58:28.652207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658127+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:58:28.658127+0000","last_peered":"2024-09-21T02:58:28.658127+0000","last_clean":"2024-09-21T02:58:28.658127+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:58:28.658127+0000","last_undegraded":"2024-09-21T02:58:28.658127+0000","last_fullsized":"2024-09-21T02:58:28.658127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648764+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:28.648764+0000","last_peered":"2024-09-21T02:58:28.648764+0000","last_clean":"2024-09-21T02:58:28.648764+0000","last_became_active":"2024-09-21T02:58:27.659069+0000","last_became_peered":"2024-09-21T02:58:27.659069+0000","last_unstale":"2024-09-21T02:58:28.648764+0000","last_undegraded":"2024-09-21T02:58:28.648764+0000","last_fullsized":"2024-09-21T02:58:28.648764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:31.780823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.656581+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:58:28.656581+0000","last_peered":"2024-09-21T02:58:28.656581+0000","last_clean":"2024-09-21T02:58:28.656581+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:58:28.656581+0000","last_undegraded":"2024-09-21T02:58:28.656581+0000","last_fullsized":"2024-09-21T02:58:28.656581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":29813,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":29,"num_read_kb":45,"num_write":61,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":217088,"data_stored":133917,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":51,"ondisk_log_size":51,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":802816,"data_stored":394943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906965,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40920,"kb_used_data":1416,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704232,"statfs":{"total":95995035648,"available":95953133568,"internally_reserved":0,"allocated":1449984,"data_stored":1014637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070490,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":395525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234015,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":395547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430243,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36724,"kb_used_data":1368,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708428,"statfs":{"total":95995035648,"available":95957430272,"internally_reserved":0,"allocated":1400832,"data_stored":985893,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593769,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":806912,"data_stored":395060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757293,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1413120,"data_stored":985610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953522,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":806912,"data_stored":394965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378686,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36184,"kb_used_data":840,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708968,"statfs":{"total":95995035648,"available":95957983232,"internally_reserved":0,"allocated":860160,"data_stored":440180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":452248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":806912,"data_stored":395033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":45237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":29209,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:58:51.291 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:58:51.549 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:51.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:51 smithi016 ceph-mon[31253]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.991+0000 7f2fa6360640 1 -- 172.21.15.16:0/304519328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0108cf0 msgr2=0x7f2fa010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.991+0000 7f2fa6360640 1 --2- 172.21.15.16:0/304519328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0108cf0 0x7f2fa010f520 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2f98009ca0 tx=0x7f2f9802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 -- 172.21.15.16:0/304519328 shutdown_connections 2024-09-21T02:58:51.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 --2- 172.21.15.16:0/304519328 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa0108cf0 0x7f2fa010f520 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 --2- 172.21.15.16:0/304519328 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa01083a0 0x7f2fa01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 --2- 172.21.15.16:0/304519328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 0x7f2fa0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 -- 172.21.15.16:0/304519328 >> 172.21.15.16:0/304519328 conn(0x7f2fa0079760 msgr2=0x7f2fa0079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 -- 172.21.15.16:0/304519328 shutdown_connections 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.992+0000 7f2fa6360640 1 -- 172.21.15.16:0/304519328 wait complete. 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 Processor -- start 2024-09-21T02:58:51.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 -- start start 2024-09-21T02:58:51.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 0x7f2fa019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:51.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa0108cf0 0x7f2fa019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.993+0000 7f2fa6360640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0113aa0 con 0x7f2fa01083a0 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2fa6360640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0113920 con 0x7f2fa0108cf0 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2fa6360640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2fa0113c20 con 0x7f2fa0102a60 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46396/0 (socket says 172.21.15.16:46396) 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/110886941 learned_addr learned my addr 172.21.15.16:0/110886941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2fa48fd640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa0108cf0 0x7f2fa019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9ffff640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 0x7f2fa019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/110886941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 msgr2=0x7f2fa019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 0x7f2fa019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/110886941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa0108cf0 msgr2=0x7f2fa019fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:51.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa0108cf0 0x7f2fa019fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 -- 172.21.15.16:0/110886941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fa01a0460 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9f7fe640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400d740 tx=0x7f2f9400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f94017070 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fa01a0750 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9400de50 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.994+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9400f620 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.995+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fa01a6eb0 con 0x7f2fa01083a0 2024-09-21T02:58:51.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.996+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2fa0103ce0 con 0x7f2fa01083a0 2024-09-21T02:58:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.997+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2f9401a070 con 0x7f2fa01083a0 2024-09-21T02:58:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.998+0000 7f2f9d7fa640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 0x7f2f880b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:51.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.998+0000 7f2f9ffff640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 0x7f2f880b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.998+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f2f940e6dd0 con 0x7f2fa01083a0 2024-09-21T02:58:52.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:51.999+0000 7f2f9ffff640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 0x7f2f880b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005d00 tx=0x7f2f9000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:52.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.001+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2f940b18a0 con 0x7f2fa01083a0 2024-09-21T02:58:52.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:51 smithi018 ceph-mon[34587]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:52.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.185+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f2fa0063800 con 0x7f2f880b65f0 2024-09-21T02:58:52.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.190+0000 7f2f9d7fa640 1 -- 172.21.15.16:0/110886941 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258612 (secure 0 0 0) 0x7f2fa0063800 con 0x7f2f880b65f0 2024-09-21T02:58:52.193 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:52.194 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 msgr2=0x7f2f880b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 0x7f2f880b8b00 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2f90005d00 tx=0x7f2f9000b410 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 msgr2=0x7f2fa019aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f2f9400d740 tx=0x7f2f9400dc00 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 shutdown_connections 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2f880b65f0 0x7f2f880b8b00 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2fa01083a0 0x7f2fa019aa90 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2fa0108cf0 0x7f2fa019fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 --2- 172.21.15.16:0/110886941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2fa0102a60 0x7f2fa019a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 >> 172.21.15.16:0/110886941 conn(0x7f2fa0079760 msgr2=0x7f2fa0106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 shutdown_connections 2024-09-21T02:58:52.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:52.194+0000 7f2fa6360640 1 -- 172.21.15.16:0/110886941 wait complete. 2024-09-21T02:58:52.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:51 smithi123 ceph-mon[34645]: from='client.14976 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:52.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:52 smithi016 ceph-mon[31253]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-21T02:58:52.929 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":272,"stamp":"2024-09-21T02:58:51.147664+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":648705,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":122,"num_read_kb":147,"num_write":204,"num_write_kb":2234,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":159,"ondisk_log_size":159,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":385664,"kb_used_data":11424,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124556160,"statfs":{"total":1151940427776,"available":1151545507840,"internally_reserved":0,"allocated":11698176,"data_stored":6685431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":55927,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":41,"num_read_kb":78,"num_write":36,"num_write_kb":70,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003302"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657798+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:58:28.657798+0000","last_peered":"2024-09-21T02:58:28.657798+0000","last_clean":"2024-09-21T02:58:28.657798+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:58:28.657798+0000","last_undegraded":"2024-09-21T02:58:28.657798+0000","last_fullsized":"2024-09-21T02:58:28.657798+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653242+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:58:28.653242+0000","last_peered":"2024-09-21T02:58:28.653242+0000","last_clean":"2024-09-21T02:58:28.653242+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:58:28.653242+0000","last_undegraded":"2024-09-21T02:58:28.653242+0000","last_fullsized":"2024-09-21T02:58:28.653242+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652869+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:58:28.652869+0000","last_peered":"2024-09-21T02:58:28.652869+0000","last_clean":"2024-09-21T02:58:28.652869+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:58:28.652869+0000","last_undegraded":"2024-09-21T02:58:28.652869+0000","last_fullsized":"2024-09-21T02:58:28.652869+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655311+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:58:28.655311+0000","last_peered":"2024-09-21T02:58:28.655311+0000","last_clean":"2024-09-21T02:58:28.655311+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:58:28.655311+0000","last_undegraded":"2024-09-21T02:58:28.655311+0000","last_fullsized":"2024-09-21T02:58:28.655311+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651117+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:58:28.651117+0000","last_peered":"2024-09-21T02:58:28.651117+0000","last_clean":"2024-09-21T02:58:28.651117+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:58:28.651117+0000","last_undegraded":"2024-09-21T02:58:28.651117+0000","last_fullsized":"2024-09-21T02:58:28.651117+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648621+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:58:28.648621+0000","last_peered":"2024-09-21T02:58:28.648621+0000","last_clean":"2024-09-21T02:58:28.648621+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:58:28.648621+0000","last_undegraded":"2024-09-21T02:58:28.648621+0000","last_fullsized":"2024-09-21T02:58:28.648621+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640584+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:58:28.640584+0000","last_peered":"2024-09-21T02:58:28.640584+0000","last_clean":"2024-09-21T02:58:28.640584+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:58:28.640584+0000","last_undegraded":"2024-09-21T02:58:28.640584+0000","last_fullsized":"2024-09-21T02:58:28.640584+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648114+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:58:28.648114+0000","last_peered":"2024-09-21T02:58:28.648114+0000","last_clean":"2024-09-21T02:58:28.648114+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:58:28.648114+0000","last_undegraded":"2024-09-21T02:58:28.648114+0000","last_fullsized":"2024-09-21T02:58:28.648114+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653175+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:58:28.653175+0000","last_peered":"2024-09-21T02:58:28.653175+0000","last_clean":"2024-09-21T02:58:28.653175+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:58:28.653175+0000","last_undegraded":"2024-09-21T02:58:28.653175+0000","last_fullsized":"2024-09-21T02:58:28.653175+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649403+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:58:28.649403+0000","last_peered":"2024-09-21T02:58:28.649403+0000","last_clean":"2024-09-21T02:58:28.649403+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:58:28.649403+0000","last_undegraded":"2024-09-21T02:58:28.649403+0000","last_fullsized":"2024-09-21T02:58:28.649403+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641259+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:58:28.641259+0000","last_peered":"2024-09-21T02:58:28.641259+0000","last_clean":"2024-09-21T02:58:28.641259+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:58:28.641259+0000","last_undegraded":"2024-09-21T02:58:28.641259+0000","last_fullsized":"2024-09-21T02:58:28.641259+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.654740+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:58:28.654740+0000","last_peered":"2024-09-21T02:58:28.654740+0000","last_clean":"2024-09-21T02:58:28.654740+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:58:28.654740+0000","last_undegraded":"2024-09-21T02:58:28.654740+0000","last_fullsized":"2024-09-21T02:58:28.654740+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639835+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:58:28.639835+0000","last_peered":"2024-09-21T02:58:28.639835+0000","last_clean":"2024-09-21T02:58:28.639835+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:58:28.639835+0000","last_undegraded":"2024-09-21T02:58:28.639835+0000","last_fullsized":"2024-09-21T02:58:28.639835+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652409+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:58:28.652409+0000","last_peered":"2024-09-21T02:58:28.652409+0000","last_clean":"2024-09-21T02:58:28.652409+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:58:28.652409+0000","last_undegraded":"2024-09-21T02:58:28.652409+0000","last_fullsized":"2024-09-21T02:58:28.652409+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652048+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:58:28.652048+0000","last_peered":"2024-09-21T02:58:28.652048+0000","last_clean":"2024-09-21T02:58:28.652048+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:58:28.652048+0000","last_undegraded":"2024-09-21T02:58:28.652048+0000","last_fullsized":"2024-09-21T02:58:28.652048+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652474+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:58:28.652474+0000","last_peered":"2024-09-21T02:58:28.652474+0000","last_clean":"2024-09-21T02:58:28.652474+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:58:28.652474+0000","last_undegraded":"2024-09-21T02:58:28.652474+0000","last_fullsized":"2024-09-21T02:58:28.652474+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657879+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:58:28.657879+0000","last_peered":"2024-09-21T02:58:28.657879+0000","last_clean":"2024-09-21T02:58:28.657879+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:58:28.657879+0000","last_undegraded":"2024-09-21T02:58:28.657879+0000","last_fullsized":"2024-09-21T02:58:28.657879+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651598+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:58:28.651598+0000","last_peered":"2024-09-21T02:58:28.651598+0000","last_clean":"2024-09-21T02:58:28.651598+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:58:28.651598+0000","last_undegraded":"2024-09-21T02:58:28.651598+0000","last_fullsized":"2024-09-21T02:58:28.651598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640396+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:58:28.640396+0000","last_peered":"2024-09-21T02:58:28.640396+0000","last_clean":"2024-09-21T02:58:28.640396+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:58:28.640396+0000","last_undegraded":"2024-09-21T02:58:28.640396+0000","last_fullsized":"2024-09-21T02:58:28.640396+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657177+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:58:28.657177+0000","last_peered":"2024-09-21T02:58:28.657177+0000","last_clean":"2024-09-21T02:58:28.657177+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:58:28.657177+0000","last_undegraded":"2024-09-21T02:58:28.657177+0000","last_fullsized":"2024-09-21T02:58:28.657177+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639873+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:58:28.639873+0000","last_peered":"2024-09-21T02:58:28.639873+0000","last_clean":"2024-09-21T02:58:28.639873+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:58:28.639873+0000","last_undegraded":"2024-09-21T02:58:28.639873+0000","last_fullsized":"2024-09-21T02:58:28.639873+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651969+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:58:28.651969+0000","last_peered":"2024-09-21T02:58:28.651969+0000","last_clean":"2024-09-21T02:58:28.651969+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:58:28.651969+0000","last_undegraded":"2024-09-21T02:58:28.651969+0000","last_fullsized":"2024-09-21T02:58:28.651969+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648273+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:58:28.648273+0000","last_peered":"2024-09-21T02:58:28.648273+0000","last_clean":"2024-09-21T02:58:28.648273+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:58:28.648273+0000","last_undegraded":"2024-09-21T02:58:28.648273+0000","last_fullsized":"2024-09-21T02:58:28.648273+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.654551+0000","last_change":"2024-09-21T02:58:27.647504+0000","last_active":"2024-09-21T02:58:28.654551+0000","last_peered":"2024-09-21T02:58:28.654551+0000","last_clean":"2024-09-21T02:58:28.654551+0000","last_became_active":"2024-09-21T02:58:27.647051+0000","last_became_peered":"2024-09-21T02:58:27.647051+0000","last_unstale":"2024-09-21T02:58:28.654551+0000","last_undegraded":"2024-09-21T02:58:28.654551+0000","last_fullsized":"2024-09-21T02:58:28.654551+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:05:21.890708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649102+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:58:28.649102+0000","last_peered":"2024-09-21T02:58:28.649102+0000","last_clean":"2024-09-21T02:58:28.649102+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:58:28.649102+0000","last_undegraded":"2024-09-21T02:58:28.649102+0000","last_fullsized":"2024-09-21T02:58:28.649102+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640473+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:58:28.640473+0000","last_peered":"2024-09-21T02:58:28.640473+0000","last_clean":"2024-09-21T02:58:28.640473+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:58:28.640473+0000","last_undegraded":"2024-09-21T02:58:28.640473+0000","last_fullsized":"2024-09-21T02:58:28.640473+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657699+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:58:28.657699+0000","last_peered":"2024-09-21T02:58:28.657699+0000","last_clean":"2024-09-21T02:58:28.657699+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:58:28.657699+0000","last_undegraded":"2024-09-21T02:58:28.657699+0000","last_fullsized":"2024-09-21T02:58:28.657699+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639385+0000","last_change":"2024-09-21T02:58:27.649283+0000","last_active":"2024-09-21T02:58:28.639385+0000","last_peered":"2024-09-21T02:58:28.639385+0000","last_clean":"2024-09-21T02:58:28.639385+0000","last_became_active":"2024-09-21T02:58:27.648919+0000","last_became_peered":"2024-09-21T02:58:27.648919+0000","last_unstale":"2024-09-21T02:58:28.639385+0000","last_undegraded":"2024-09-21T02:58:28.639385+0000","last_fullsized":"2024-09-21T02:58:28.639385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:57:27.291907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653438+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:58:28.653438+0000","last_peered":"2024-09-21T02:58:28.653438+0000","last_clean":"2024-09-21T02:58:28.653438+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:58:28.653438+0000","last_undegraded":"2024-09-21T02:58:28.653438+0000","last_fullsized":"2024-09-21T02:58:28.653438+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640008+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:58:28.640008+0000","last_peered":"2024-09-21T02:58:28.640008+0000","last_clean":"2024-09-21T02:58:28.640008+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:58:28.640008+0000","last_undegraded":"2024-09-21T02:58:28.640008+0000","last_fullsized":"2024-09-21T02:58:28.640008+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640021+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:58:28.640021+0000","last_peered":"2024-09-21T02:58:28.640021+0000","last_clean":"2024-09-21T02:58:28.640021+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:58:28.640021+0000","last_undegraded":"2024-09-21T02:58:28.640021+0000","last_fullsized":"2024-09-21T02:58:28.640021+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658161+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:58:28.658161+0000","last_peered":"2024-09-21T02:58:28.658161+0000","last_clean":"2024-09-21T02:58:28.658161+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:58:28.658161+0000","last_undegraded":"2024-09-21T02:58:28.658161+0000","last_fullsized":"2024-09-21T02:58:28.658161+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651265+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:58:28.651265+0000","last_peered":"2024-09-21T02:58:28.651265+0000","last_clean":"2024-09-21T02:58:28.651265+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:58:28.651265+0000","last_undegraded":"2024-09-21T02:58:28.651265+0000","last_fullsized":"2024-09-21T02:58:28.651265+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652418+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:58:28.652418+0000","last_peered":"2024-09-21T02:58:28.652418+0000","last_clean":"2024-09-21T02:58:28.652418+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:58:28.652418+0000","last_undegraded":"2024-09-21T02:58:28.652418+0000","last_fullsized":"2024-09-21T02:58:28.652418+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652160+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:58:28.652160+0000","last_peered":"2024-09-21T02:58:28.652160+0000","last_clean":"2024-09-21T02:58:28.652160+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:58:28.652160+0000","last_undegraded":"2024-09-21T02:58:28.652160+0000","last_fullsized":"2024-09-21T02:58:28.652160+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655294+0000","last_change":"2024-09-21T02:58:25.638302+0000","last_active":"2024-09-21T02:58:28.655294+0000","last_peered":"2024-09-21T02:58:28.655294+0000","last_clean":"2024-09-21T02:58:28.655294+0000","last_became_active":"2024-09-21T02:58:25.637899+0000","last_became_peered":"2024-09-21T02:58:25.637899+0000","last_unstale":"2024-09-21T02:58:28.655294+0000","last_undegraded":"2024-09-21T02:58:28.655294+0000","last_fullsized":"2024-09-21T02:58:28.655294+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:38:57.896986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639186+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:58:28.639186+0000","last_peered":"2024-09-21T02:58:28.639186+0000","last_clean":"2024-09-21T02:58:28.639186+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:58:28.639186+0000","last_undegraded":"2024-09-21T02:58:28.639186+0000","last_fullsized":"2024-09-21T02:58:28.639186+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648498+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:58:28.648498+0000","last_peered":"2024-09-21T02:58:28.648498+0000","last_clean":"2024-09-21T02:58:28.648498+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:58:28.648498+0000","last_undegraded":"2024-09-21T02:58:28.648498+0000","last_fullsized":"2024-09-21T02:58:28.648498+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653445+0000","last_change":"2024-09-21T02:58:25.640688+0000","last_active":"2024-09-21T02:58:28.653445+0000","last_peered":"2024-09-21T02:58:28.653445+0000","last_clean":"2024-09-21T02:58:28.653445+0000","last_became_active":"2024-09-21T02:58:25.640306+0000","last_became_peered":"2024-09-21T02:58:25.640306+0000","last_unstale":"2024-09-21T02:58:28.653445+0000","last_undegraded":"2024-09-21T02:58:28.653445+0000","last_fullsized":"2024-09-21T02:58:28.653445+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:04.808100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651262+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:58:28.651262+0000","last_peered":"2024-09-21T02:58:28.651262+0000","last_clean":"2024-09-21T02:58:28.651262+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:58:28.651262+0000","last_undegraded":"2024-09-21T02:58:28.651262+0000","last_fullsized":"2024-09-21T02:58:28.651262+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648932+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:58:28.648932+0000","last_peered":"2024-09-21T02:58:28.648932+0000","last_clean":"2024-09-21T02:58:28.648932+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:58:28.648932+0000","last_undegraded":"2024-09-21T02:58:28.648932+0000","last_fullsized":"2024-09-21T02:58:28.648932+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653072+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:58:28.653072+0000","last_peered":"2024-09-21T02:58:28.653072+0000","last_clean":"2024-09-21T02:58:28.653072+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:58:28.653072+0000","last_undegraded":"2024-09-21T02:58:28.653072+0000","last_fullsized":"2024-09-21T02:58:28.653072+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640941+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:58:28.640941+0000","last_peered":"2024-09-21T02:58:28.640941+0000","last_clean":"2024-09-21T02:58:28.640941+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:58:28.640941+0000","last_undegraded":"2024-09-21T02:58:28.640941+0000","last_fullsized":"2024-09-21T02:58:28.640941+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652508+0000","last_change":"2024-09-21T02:58:27.655900+0000","last_active":"2024-09-21T02:58:28.652508+0000","last_peered":"2024-09-21T02:58:28.652508+0000","last_clean":"2024-09-21T02:58:28.652508+0000","last_became_active":"2024-09-21T02:58:27.655522+0000","last_became_peered":"2024-09-21T02:58:27.655522+0000","last_unstale":"2024-09-21T02:58:28.652508+0000","last_undegraded":"2024-09-21T02:58:28.652508+0000","last_fullsized":"2024-09-21T02:58:28.652508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:31:58.361260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659590+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:58:28.659590+0000","last_peered":"2024-09-21T02:58:28.659590+0000","last_clean":"2024-09-21T02:58:28.659590+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:58:28.659590+0000","last_undegraded":"2024-09-21T02:58:28.659590+0000","last_fullsized":"2024-09-21T02:58:28.659590+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653794+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:58:28.653794+0000","last_peered":"2024-09-21T02:58:28.653794+0000","last_clean":"2024-09-21T02:58:28.653794+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:58:28.653794+0000","last_undegraded":"2024-09-21T02:58:28.653794+0000","last_fullsized":"2024-09-21T02:58:28.653794+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"75'25","reported_seq":79,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:48.705751+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T02:58:48.705751+0000","last_peered":"2024-09-21T02:58:48.705751+0000","last_clean":"2024-09-21T02:58:48.705751+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T02:58:48.705751+0000","last_undegraded":"2024-09-21T02:58:48.705751+0000","last_fullsized":"2024-09-21T02:58:48.705751+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":25,"log_dups_size":0,"ondisk_log_size":25,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":57305,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":72,"num_write":25,"num_write_kb":66,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651717+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:58:28.651717+0000","last_peered":"2024-09-21T02:58:28.651717+0000","last_clean":"2024-09-21T02:58:28.651717+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:58:28.651717+0000","last_undegraded":"2024-09-21T02:58:28.651717+0000","last_fullsized":"2024-09-21T02:58:28.651717+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649247+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:58:28.649247+0000","last_peered":"2024-09-21T02:58:28.649247+0000","last_clean":"2024-09-21T02:58:28.649247+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:58:28.649247+0000","last_undegraded":"2024-09-21T02:58:28.649247+0000","last_fullsized":"2024-09-21T02:58:28.649247+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639467+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:58:28.639467+0000","last_peered":"2024-09-21T02:58:28.639467+0000","last_clean":"2024-09-21T02:58:28.639467+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:58:28.639467+0000","last_undegraded":"2024-09-21T02:58:28.639467+0000","last_fullsized":"2024-09-21T02:58:28.639467+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651077+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:58:28.651077+0000","last_peered":"2024-09-21T02:58:28.651077+0000","last_clean":"2024-09-21T02:58:28.651077+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:58:28.651077+0000","last_undegraded":"2024-09-21T02:58:28.651077+0000","last_fullsized":"2024-09-21T02:58:28.651077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641137+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:58:28.641137+0000","last_peered":"2024-09-21T02:58:28.641137+0000","last_clean":"2024-09-21T02:58:28.641137+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:58:28.641137+0000","last_undegraded":"2024-09-21T02:58:28.641137+0000","last_fullsized":"2024-09-21T02:58:28.641137+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652627+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:58:28.652627+0000","last_peered":"2024-09-21T02:58:28.652627+0000","last_clean":"2024-09-21T02:58:28.652627+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:58:28.652627+0000","last_undegraded":"2024-09-21T02:58:28.652627+0000","last_fullsized":"2024-09-21T02:58:28.652627+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640595+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:58:28.640595+0000","last_peered":"2024-09-21T02:58:28.640595+0000","last_clean":"2024-09-21T02:58:28.640595+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:58:28.640595+0000","last_undegraded":"2024-09-21T02:58:28.640595+0000","last_fullsized":"2024-09-21T02:58:28.640595+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659030+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:58:28.659030+0000","last_peered":"2024-09-21T02:58:28.659030+0000","last_clean":"2024-09-21T02:58:28.659030+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:58:28.659030+0000","last_undegraded":"2024-09-21T02:58:28.659030+0000","last_fullsized":"2024-09-21T02:58:28.659030+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651476+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:58:28.651476+0000","last_peered":"2024-09-21T02:58:28.651476+0000","last_clean":"2024-09-21T02:58:28.651476+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:58:28.651476+0000","last_undegraded":"2024-09-21T02:58:28.651476+0000","last_fullsized":"2024-09-21T02:58:28.651476+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658826+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:58:28.658826+0000","last_peered":"2024-09-21T02:58:28.658826+0000","last_clean":"2024-09-21T02:58:28.658826+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:58:28.658826+0000","last_undegraded":"2024-09-21T02:58:28.658826+0000","last_fullsized":"2024-09-21T02:58:28.658826+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.657319+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:58:28.657319+0000","last_peered":"2024-09-21T02:58:28.657319+0000","last_clean":"2024-09-21T02:58:28.657319+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:58:28.657319+0000","last_undegraded":"2024-09-21T02:58:28.657319+0000","last_fullsized":"2024-09-21T02:58:28.657319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.647905+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:58:28.647905+0000","last_peered":"2024-09-21T02:58:28.647905+0000","last_clean":"2024-09-21T02:58:28.647905+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:58:28.647905+0000","last_undegraded":"2024-09-21T02:58:28.647905+0000","last_fullsized":"2024-09-21T02:58:28.647905+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640947+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:58:28.640947+0000","last_peered":"2024-09-21T02:58:28.640947+0000","last_clean":"2024-09-21T02:58:28.640947+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:58:28.640947+0000","last_undegraded":"2024-09-21T02:58:28.640947+0000","last_fullsized":"2024-09-21T02:58:28.640947+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.659668+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:58:28.659668+0000","last_peered":"2024-09-21T02:58:28.659668+0000","last_clean":"2024-09-21T02:58:28.659668+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:58:28.659668+0000","last_undegraded":"2024-09-21T02:58:28.659668+0000","last_fullsized":"2024-09-21T02:58:28.659668+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:29.459725+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:58:29.459725+0000","last_peered":"2024-09-21T02:58:29.459725+0000","last_clean":"2024-09-21T02:58:29.459725+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:58:29.459725+0000","last_undegraded":"2024-09-21T02:58:29.459725+0000","last_fullsized":"2024-09-21T02:58:29.459725+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640370+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:58:28.640370+0000","last_peered":"2024-09-21T02:58:28.640370+0000","last_clean":"2024-09-21T02:58:28.640370+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:58:28.640370+0000","last_undegraded":"2024-09-21T02:58:28.640370+0000","last_fullsized":"2024-09-21T02:58:28.640370+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641410+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:58:28.641410+0000","last_peered":"2024-09-21T02:58:28.641410+0000","last_clean":"2024-09-21T02:58:28.641410+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:58:28.641410+0000","last_undegraded":"2024-09-21T02:58:28.641410+0000","last_fullsized":"2024-09-21T02:58:28.641410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":89,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640685+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:58:28.640685+0000","last_peered":"2024-09-21T02:58:28.640685+0000","last_clean":"2024-09-21T02:58:28.640685+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:58:28.640685+0000","last_undegraded":"2024-09-21T02:58:28.640685+0000","last_fullsized":"2024-09-21T02:58:28.640685+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653863+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:58:28.653863+0000","last_peered":"2024-09-21T02:58:28.653863+0000","last_clean":"2024-09-21T02:58:28.653863+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:58:28.653863+0000","last_undegraded":"2024-09-21T02:58:28.653863+0000","last_fullsized":"2024-09-21T02:58:28.653863+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648633+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:58:28.648633+0000","last_peered":"2024-09-21T02:58:28.648633+0000","last_clean":"2024-09-21T02:58:28.648633+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:58:28.648633+0000","last_undegraded":"2024-09-21T02:58:28.648633+0000","last_fullsized":"2024-09-21T02:58:28.648633+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640683+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:58:28.640683+0000","last_peered":"2024-09-21T02:58:28.640683+0000","last_clean":"2024-09-21T02:58:28.640683+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:58:28.640683+0000","last_undegraded":"2024-09-21T02:58:28.640683+0000","last_fullsized":"2024-09-21T02:58:28.640683+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639221+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:28.639221+0000","last_peered":"2024-09-21T02:58:28.639221+0000","last_clean":"2024-09-21T02:58:28.639221+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:58:28.639221+0000","last_undegraded":"2024-09-21T02:58:28.639221+0000","last_fullsized":"2024-09-21T02:58:28.639221+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639702+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:58:28.639702+0000","last_peered":"2024-09-21T02:58:28.639702+0000","last_clean":"2024-09-21T02:58:28.639702+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:58:28.639702+0000","last_undegraded":"2024-09-21T02:58:28.639702+0000","last_fullsized":"2024-09-21T02:58:28.639702+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653188+0000","last_change":"2024-09-21T02:58:27.658246+0000","last_active":"2024-09-21T02:58:28.653188+0000","last_peered":"2024-09-21T02:58:28.653188+0000","last_clean":"2024-09-21T02:58:28.653188+0000","last_became_active":"2024-09-21T02:58:27.657143+0000","last_became_peered":"2024-09-21T02:58:27.657143+0000","last_unstale":"2024-09-21T02:58:28.653188+0000","last_undegraded":"2024-09-21T02:58:28.653188+0000","last_fullsized":"2024-09-21T02:58:28.653188+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:14:56.282781+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.641359+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:58:28.641359+0000","last_peered":"2024-09-21T02:58:28.641359+0000","last_clean":"2024-09-21T02:58:28.641359+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:58:28.641359+0000","last_undegraded":"2024-09-21T02:58:28.641359+0000","last_fullsized":"2024-09-21T02:58:28.641359+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649070+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:58:28.649070+0000","last_peered":"2024-09-21T02:58:28.649070+0000","last_clean":"2024-09-21T02:58:28.649070+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:58:28.649070+0000","last_undegraded":"2024-09-21T02:58:28.649070+0000","last_fullsized":"2024-09-21T02:58:28.649070+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651315+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:58:28.651315+0000","last_peered":"2024-09-21T02:58:28.651315+0000","last_clean":"2024-09-21T02:58:28.651315+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:58:28.651315+0000","last_undegraded":"2024-09-21T02:58:28.651315+0000","last_fullsized":"2024-09-21T02:58:28.651315+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":15,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:29.187908+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:58:29.187908+0000","last_peered":"2024-09-21T02:58:29.187908+0000","last_clean":"2024-09-21T02:58:29.187908+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:58:29.187908+0000","last_undegraded":"2024-09-21T02:58:29.187908+0000","last_fullsized":"2024-09-21T02:58:29.187908+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.651342+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:58:28.651342+0000","last_peered":"2024-09-21T02:58:28.651342+0000","last_clean":"2024-09-21T02:58:28.651342+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:58:28.651342+0000","last_undegraded":"2024-09-21T02:58:28.651342+0000","last_fullsized":"2024-09-21T02:58:28.651342+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639623+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:58:28.639623+0000","last_peered":"2024-09-21T02:58:28.639623+0000","last_clean":"2024-09-21T02:58:28.639623+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:58:28.639623+0000","last_undegraded":"2024-09-21T02:58:28.639623+0000","last_fullsized":"2024-09-21T02:58:28.639623+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'8","reported_seq":27,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:48.647833+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:58:48.647833+0000","last_peered":"2024-09-21T02:58:48.647833+0000","last_clean":"2024-09-21T02:58:48.647833+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:58:48.647833+0000","last_undegraded":"2024-09-21T02:58:48.647833+0000","last_fullsized":"2024-09-21T02:58:48.647833+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":16,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.649252+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:58:28.649252+0000","last_peered":"2024-09-21T02:58:28.649252+0000","last_clean":"2024-09-21T02:58:28.649252+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:58:28.649252+0000","last_undegraded":"2024-09-21T02:58:28.649252+0000","last_fullsized":"2024-09-21T02:58:28.649252+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.656897+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:58:28.656897+0000","last_peered":"2024-09-21T02:58:28.656897+0000","last_clean":"2024-09-21T02:58:28.656897+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:58:28.656897+0000","last_undegraded":"2024-09-21T02:58:28.656897+0000","last_fullsized":"2024-09-21T02:58:28.656897+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653499+0000","last_change":"2024-09-21T02:58:27.656418+0000","last_active":"2024-09-21T02:58:28.653499+0000","last_peered":"2024-09-21T02:58:28.653499+0000","last_clean":"2024-09-21T02:58:28.653499+0000","last_became_active":"2024-09-21T02:58:27.656004+0000","last_became_peered":"2024-09-21T02:58:27.656004+0000","last_unstale":"2024-09-21T02:58:28.653499+0000","last_undegraded":"2024-09-21T02:58:28.653499+0000","last_fullsized":"2024-09-21T02:58:28.653499+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:44:57.543580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652618+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:58:28.652618+0000","last_peered":"2024-09-21T02:58:28.652618+0000","last_clean":"2024-09-21T02:58:28.652618+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:58:28.652618+0000","last_undegraded":"2024-09-21T02:58:28.652618+0000","last_fullsized":"2024-09-21T02:58:28.652618+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.650543+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:58:28.650543+0000","last_peered":"2024-09-21T02:58:28.650543+0000","last_clean":"2024-09-21T02:58:28.650543+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:58:28.650543+0000","last_undegraded":"2024-09-21T02:58:28.650543+0000","last_fullsized":"2024-09-21T02:58:28.650543+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648456+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:58:28.648456+0000","last_peered":"2024-09-21T02:58:28.648456+0000","last_clean":"2024-09-21T02:58:28.648456+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:58:28.648456+0000","last_undegraded":"2024-09-21T02:58:28.648456+0000","last_fullsized":"2024-09-21T02:58:28.648456+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658390+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:58:28.658390+0000","last_peered":"2024-09-21T02:58:28.658390+0000","last_clean":"2024-09-21T02:58:28.658390+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:58:28.658390+0000","last_undegraded":"2024-09-21T02:58:28.658390+0000","last_fullsized":"2024-09-21T02:58:28.658390+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":19,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.639964+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:58:28.639964+0000","last_peered":"2024-09-21T02:58:28.639964+0000","last_clean":"2024-09-21T02:58:28.639964+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:58:28.639964+0000","last_undegraded":"2024-09-21T02:58:28.639964+0000","last_fullsized":"2024-09-21T02:58:28.639964+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648778+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:58:28.648778+0000","last_peered":"2024-09-21T02:58:28.648778+0000","last_clean":"2024-09-21T02:58:28.648778+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:58:28.648778+0000","last_undegraded":"2024-09-21T02:58:28.648778+0000","last_fullsized":"2024-09-21T02:58:28.648778+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":22,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655405+0000","last_change":"2024-09-21T02:58:25.640895+0000","last_active":"2024-09-21T02:58:28.655405+0000","last_peered":"2024-09-21T02:58:28.655405+0000","last_clean":"2024-09-21T02:58:28.655405+0000","last_became_active":"2024-09-21T02:58:25.640488+0000","last_became_peered":"2024-09-21T02:58:25.640488+0000","last_unstale":"2024-09-21T02:58:28.655405+0000","last_undegraded":"2024-09-21T02:58:28.655405+0000","last_fullsized":"2024-09-21T02:58:28.655405+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:35:55.598375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.640922+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:58:28.640922+0000","last_peered":"2024-09-21T02:58:28.640922+0000","last_clean":"2024-09-21T02:58:28.640922+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:58:28.640922+0000","last_undegraded":"2024-09-21T02:58:28.640922+0000","last_fullsized":"2024-09-21T02:58:28.640922+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653327+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:58:28.653327+0000","last_peered":"2024-09-21T02:58:28.653327+0000","last_clean":"2024-09-21T02:58:28.653327+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:58:28.653327+0000","last_undegraded":"2024-09-21T02:58:28.653327+0000","last_fullsized":"2024-09-21T02:58:28.653327+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658913+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:58:28.658913+0000","last_peered":"2024-09-21T02:58:28.658913+0000","last_clean":"2024-09-21T02:58:28.658913+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:58:28.658913+0000","last_undegraded":"2024-09-21T02:58:28.658913+0000","last_fullsized":"2024-09-21T02:58:28.658913+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.653465+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:58:28.653465+0000","last_peered":"2024-09-21T02:58:28.653465+0000","last_clean":"2024-09-21T02:58:28.653465+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:58:28.653465+0000","last_undegraded":"2024-09-21T02:58:28.653465+0000","last_fullsized":"2024-09-21T02:58:28.653465+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.655188+0000","last_change":"2024-09-21T02:58:25.642383+0000","last_active":"2024-09-21T02:58:28.655188+0000","last_peered":"2024-09-21T02:58:28.655188+0000","last_clean":"2024-09-21T02:58:28.655188+0000","last_became_active":"2024-09-21T02:58:25.641933+0000","last_became_peered":"2024-09-21T02:58:25.641933+0000","last_unstale":"2024-09-21T02:58:28.655188+0000","last_undegraded":"2024-09-21T02:58:28.655188+0000","last_fullsized":"2024-09-21T02:58:28.655188+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:12:34.484094+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.652207+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:58:28.652207+0000","last_peered":"2024-09-21T02:58:28.652207+0000","last_clean":"2024-09-21T02:58:28.652207+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:58:28.652207+0000","last_undegraded":"2024-09-21T02:58:28.652207+0000","last_fullsized":"2024-09-21T02:58:28.652207+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":20,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.658127+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:58:28.658127+0000","last_peered":"2024-09-21T02:58:28.658127+0000","last_clean":"2024-09-21T02:58:28.658127+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:58:28.658127+0000","last_undegraded":"2024-09-21T02:58:28.658127+0000","last_fullsized":"2024-09-21T02:58:28.658127+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":14,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.648764+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:28.648764+0000","last_peered":"2024-09-21T02:58:28.648764+0000","last_clean":"2024-09-21T02:58:28.648764+0000","last_became_active":"2024-09-21T02:58:27.659069+0000","last_became_peered":"2024-09-21T02:58:27.659069+0000","last_unstale":"2024-09-21T02:58:28.648764+0000","last_undegraded":"2024-09-21T02:58:28.648764+0000","last_fullsized":"2024-09-21T02:58:28.648764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:43:31.780823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":24,"reported_epoch":75,"state":"active+clean","last_fresh":"2024-09-21T02:58:28.656581+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:58:28.656581+0000","last_peered":"2024-09-21T02:58:28.656581+0000","last_clean":"2024-09-21T02:58:28.656581+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:58:28.656581+0000","last_undegraded":"2024-09-21T02:58:28.656581+0000","last_fullsized":"2024-09-21T02:58:28.656581+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":58103,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":41,"num_read_kb":78,"num_write":79,"num_write_kb":82,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":174309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":65,"ondisk_log_size":65,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743440,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":784,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":802816,"data_stored":394943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906966,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40948,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704204,"statfs":{"total":95995035648,"available":95953104896,"internally_reserved":0,"allocated":1478656,"data_stored":1042927,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070491,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27812,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717340,"statfs":{"total":95995035648,"available":95966556160,"internally_reserved":0,"allocated":806912,"data_stored":395525,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234016,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27816,"kb_used_data":792,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717336,"statfs":{"total":95995035648,"available":95966552064,"internally_reserved":0,"allocated":811008,"data_stored":395547,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430244,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36716,"kb_used_data":1372,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708436,"statfs":{"total":95995035648,"available":95957438464,"internally_reserved":0,"allocated":1404928,"data_stored":986010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593770,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36132,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709020,"statfs":{"total":95995035648,"available":95958036480,"internally_reserved":0,"allocated":806912,"data_stored":395060,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757294,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32564,"kb_used_data":1380,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712588,"statfs":{"total":95995035648,"available":95961690112,"internally_reserved":0,"allocated":1413120,"data_stored":985610,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953523,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36144,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93709008,"statfs":{"total":95995035648,"available":95958024192,"internally_reserved":0,"allocated":806912,"data_stored":394965,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149752,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27824,"kb_used_data":800,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717328,"statfs":{"total":95995035648,"available":95966543872,"internally_reserved":0,"allocated":819200,"data_stored":395281,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378687,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36196,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708956,"statfs":{"total":95995035648,"available":95957970944,"internally_reserved":0,"allocated":872448,"data_stored":452282,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542210,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27872,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717280,"statfs":{"total":95995035648,"available":95966494720,"internally_reserved":0,"allocated":868352,"data_stored":452248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705736,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27820,"kb_used_data":788,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717332,"statfs":{"total":95995035648,"available":95966547968,"internally_reserved":0,"allocated":806912,"data_stored":395033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":57305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":69632,"data_stored":57339,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":57499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:58:52.933 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:58:52.933 INFO:tasks.cephadm:Setting up client nodes... 2024-09-21T02:58:52.934 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-09-21T02:58:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:52 smithi018 ceph-mon[34587]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-21T02:58:53.196 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:52 smithi123 ceph-mon[34645]: pgmap v272: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 5.8 KiB/s wr, 6 op/s 2024-09-21T02:58:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.658+0000 7f501ca00640 1 -- 172.21.15.16:0/814140834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50181050e0 msgr2=0x7f501810d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:53.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.658+0000 7f501ca00640 1 --2- 172.21.15.16:0/814140834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50181050e0 0x7f501810d720 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f500c009900 tx=0x7f500c02a3f0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 -- 172.21.15.16:0/814140834 shutdown_connections 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 --2- 172.21.15.16:0/814140834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50181050e0 0x7f501810d720 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 --2- 172.21.15.16:0/814140834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f501810dc60 0x7f5018110130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 --2- 172.21.15.16:0/814140834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f5018104ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 -- 172.21.15.16:0/814140834 >> 172.21.15.16:0/814140834 conn(0x7f50180fe2e0 msgr2=0x7f5018100730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 -- 172.21.15.16:0/814140834 shutdown_connections 2024-09-21T02:58:53.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 -- 172.21.15.16:0/814140834 wait complete. 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.659+0000 7f501ca00640 1 Processor -- start 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 -- start start 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f501819a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50181050e0 0x7f501819a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50181138f0 con 0x7f501810dc60 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5018113770 con 0x7f50181050e0 2024-09-21T02:58:53.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501ca00640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5018113a70 con 0x7f50181046b0 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46412/0 (socket says 172.21.15.16:46412) 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501759e640 1 -- 172.21.15.16:0/1450608926 learned_addr learned my addr 172.21.15.16:0/1450608926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f501659c640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50181050e0 0x7f501819a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.660+0000 7f5016d9d640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f501819a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 -- 172.21.15.16:0/1450608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 msgr2=0x7f501819a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f501819a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 -- 172.21.15.16:0/1450608926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50181050e0 msgr2=0x7f501819a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50181050e0 0x7f501819a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 -- 172.21.15.16:0/1450608926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50181a0140 con 0x7f501810dc60 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f5016d9d640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f501819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501759e640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f501000cce0 tx=0x7f501000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f501000f450 con 0x7f501810dc60 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50181a0430 con 0x7f501810dc60 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f501000faa0 con 0x7f501810dc60 2024-09-21T02:58:53.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5010018b70 con 0x7f501810dc60 2024-09-21T02:58:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.661+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50181a6c50 con 0x7f501810dc60 2024-09-21T02:58:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.662+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5018104ac0 con 0x7f501810dc60 2024-09-21T02:58:53.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.664+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f5010018d10 con 0x7f501810dc60 2024-09-21T02:58:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.664+0000 7f4ffffff640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 0x7f50000b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.665+0000 7f5016d9d640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 0x7f50000b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.665+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(75..75 src has 1..75) ==== 8194+0+0 (secure 0 0 0) 0x7f50100e78d0 con 0x7f501810dc60 2024-09-21T02:58:53.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.665+0000 7f5016d9d640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 0x7f50000b8b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f500800abe0 tx=0x7f500800a5c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:53.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.669+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5010014030 con 0x7f501810dc60 2024-09-21T02:58:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.937+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f50181024a0 con 0x7f501810dc60 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-21T02:58:53.939 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:53 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:58:53.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.940+0000 7f4ffffff640 1 -- 172.21.15.16:0/1450608926 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f50100ae020 con 0x7f501810dc60 2024-09-21T02:58:53.941 INFO:teuthology.orchestra.run.smithi016.stdout:[client.0] 2024-09-21T02:58:53.941 INFO:teuthology.orchestra.run.smithi016.stdout: key = AQBtNu5moE70NxAAnR4X5i31lRqEsoPx77GTqA== 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 msgr2=0x7f50000b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 0x7f50000b8b20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f500800abe0 tx=0x7f500800a5c0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 msgr2=0x7f501819fad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f501000cce0 tx=0x7f501000a620 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 shutdown_connections 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50000b6610 0x7f50000b8b20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f501810dc60 0x7f501819fad0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50181050e0 0x7f501819a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 --2- 172.21.15.16:0/1450608926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50181046b0 0x7f501819a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.942+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 >> 172.21.15.16:0/1450608926 conn(0x7f50180fe2e0 msgr2=0x7f5018107740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:53.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.943+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 shutdown_connections 2024-09-21T02:58:53.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:53.943+0000 7f501ca00640 1 -- 172.21.15.16:0/1450608926 wait complete. 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T02:58:54.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-21T02:58:54.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T02:58:54.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T02:58:54.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-21T02:58:54.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:53 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:58:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='client.14982 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:58:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]} : dispatch 2024-09-21T02:58:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]} : dispatch 2024-09-21T02:58:54.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]} : dispatch 2024-09-21T02:58:54.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:53 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]} : dispatch 2024-09-21T02:58:54.681 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T02:58:54.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-09-21T02:58:54.681 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-09-21T02:58:54.720 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph config log 1 --format=json 2024-09-21T02:58:55.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:58:55.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:54 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:58:55.018 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T02:58:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T02:58:55.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-21T02:58:55.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:58:55.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:58:55.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:58:55.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:54 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:58:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: pgmap v273: 97 pgs: 97 active+clean; 634 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 5.7 KiB/s wr, 4 op/s 2024-09-21T02:58:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.b", "id": [2, 1]}]': finished 2024-09-21T02:58:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1a", "id": [8, 4]}]': finished 2024-09-21T02:58:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1d", "id": [2, 4, 8, 11]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [5, 9]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.6", "id": [4, 6]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.17", "id": [4, 8]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.1e", "id": [3, 9]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.24", "id": [4, 9]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.28", "id": [3, 8]}]': finished 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: osdmap e76: 12 total, 12 up, 12 in 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd={"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} : dispatch 2024-09-21T02:58:55.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:54 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1450608926' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-09-21T02:58:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 -- 172.21.15.16:0/4212350668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6481049e0 msgr2=0x7fe648104df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:55.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/4212350668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6481049e0 0x7fe648104df0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fe638009900 tx=0x7fe63802a410 comp rx=0 tx=0).stop 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 -- 172.21.15.16:0/4212350668 shutdown_connections 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/4212350668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 0x7fe648110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/4212350668 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe648105330 0x7fe64810d880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/4212350668 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6481049e0 0x7fe648104df0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 -- 172.21.15.16:0/4212350668 >> 172.21.15.16:0/4212350668 conn(0x7fe648079720 msgr2=0x7fe648079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 -- 172.21.15.16:0/4212350668 shutdown_connections 2024-09-21T02:58:55.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.519+0000 7fe64e8eb640 1 -- 172.21.15.16:0/4212350668 wait complete. 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 Processor -- start 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 -- start start 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 0x7fe648102720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 0x7fe6481031a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe648113ae0 con 0x7fe648105330 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe648113960 con 0x7fe6481049e0 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe64e8eb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe648113c60 con 0x7fe64810de40 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe6477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe6477fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37574/0 (socket says 172.21.15.16:37574) 2024-09-21T02:58:55.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe6477fe640 1 -- 172.21.15.16:0/581101904 learned_addr learned my addr 172.21.15.16:0/581101904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.520+0000 7fe647fff640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 0x7fe648102720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe64ce88640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 0x7fe6481031a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 -- 172.21.15.16:0/581101904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 msgr2=0x7fe6481031a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 0x7fe6481031a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 -- 172.21.15.16:0/581101904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 msgr2=0x7fe648102720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 0x7fe648102720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 -- 172.21.15.16:0/581101904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6481a8790 con 0x7fe648105330 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe647fff640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 0x7fe648102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6477fe640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c970 tx=0x7fe63c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004220 con 0x7fe648105330 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6481a8a80 con 0x7fe648105330 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe63c0043c0 con 0x7fe648105330 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe63c004a00 con 0x7fe648105330 2024-09-21T02:58:55.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.521+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6481a8f70 con 0x7fe648105330 2024-09-21T02:58:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.522+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe64806c240 con 0x7fe648105330 2024-09-21T02:58:55.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.524+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe63c0079b0 con 0x7fe648105330 2024-09-21T02:58:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.524+0000 7fe6457fa640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 0x7fe6300b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.525+0000 7fe647fff640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 0x7fe6300b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.525+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fe63c0e7840 con 0x7fe648105330 2024-09-21T02:58:55.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.525+0000 7fe647fff640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 0x7fe6300b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:55.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.528+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe63c014030 con 0x7fe648105330 2024-09-21T02:58:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.715+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "config log", "num": 1, "format": "json"} v 0) -- 0x7fe648103a20 con 0x7fe648105330 2024-09-21T02:58:55.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.716+0000 7fe6457fa640 1 -- 172.21.15.16:0/581101904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "config log", "num": 1, "format": "json"}]=0 v56) ==== 86+0+133 (secure 0 0 0) 0x7fe648103a20 con 0x7fe648105330 2024-09-21T02:58:55.717 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 msgr2=0x7fe6300b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 0x7fe6300b8b00 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fe638000c00 tx=0x7fe638009680 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 msgr2=0x7fe648102c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fe63c00c970 tx=0x7fe63c00ce30 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 shutdown_connections 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe6300b65f0 0x7fe6300b8b00 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe648105330 0x7fe648102c60 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe64810de40 0x7fe6481031a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 --2- 172.21.15.16:0/581101904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6481049e0 0x7fe648102720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 >> 172.21.15.16:0/581101904 conn(0x7fe648079720 msgr2=0x7fe6481078f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 shutdown_connections 2024-09-21T02:58:55.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:55.718+0000 7fe64e8eb640 1 -- 172.21.15.16:0/581101904 wait complete. 2024-09-21T02:58:56.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:55 smithi016 ceph-mon[31253]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:58:56.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:55 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/581101904' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:58:56.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:55 smithi018 ceph-mon[34587]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:58:56.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:55 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/581101904' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:58:56.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:55 smithi123 ceph-mon[34645]: osdmap e77: 12 total, 12 up, 12 in 2024-09-21T02:58:56.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:55 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/581101904' entity='client.admin' cmd={"prefix": "config log", "num": 1, "format": "json"} : dispatch 2024-09-21T02:58:56.476 INFO:teuthology.orchestra.run.smithi016.stdout:[{"version":56,"timestamp":"2024-09-21T02:58:17.885906+0000","name":"","changes":[{"name":"mds.all/mds_join_fs","new_value":"all"}]}] 2024-09-21T02:58:56.476 INFO:tasks.ceph_manager:config epoch is 56 2024-09-21T02:58:56.476 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-09-21T02:58:56.477 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-09-21T02:58:56.477 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph mgr dump --format=json 2024-09-21T02:58:56.743 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:57.004 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:56 smithi016 ceph-mon[31253]: pgmap v276: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%) 2024-09-21T02:58:57.005 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:56 smithi016 ceph-mon[31253]: Health check failed: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded (PG_DEGRADED) 2024-09-21T02:58:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:56 smithi018 ceph-mon[34587]: pgmap v276: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%) 2024-09-21T02:58:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:56 smithi018 ceph-mon[34587]: Health check failed: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded (PG_DEGRADED) 2024-09-21T02:58:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:56 smithi123 ceph-mon[34645]: pgmap v276: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%) 2024-09-21T02:58:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:56 smithi123 ceph-mon[34645]: Health check failed: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded (PG_DEGRADED) 2024-09-21T02:58:57.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 -- 172.21.15.16:0/233308961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04181049b0 msgr2=0x7f0418104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 --2- 172.21.15.16:0/233308961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04181049b0 0x7f0418104dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0414009900 tx=0x7f041402a410 comp rx=0 tx=0).stop 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 -- 172.21.15.16:0/233308961 shutdown_connections 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 --2- 172.21.15.16:0/233308961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f041810de10 0x7f0418110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 --2- 172.21.15.16:0/233308961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0418105300 0x7f041810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 --2- 172.21.15.16:0/233308961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f04181049b0 0x7f0418104dc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 -- 172.21.15.16:0/233308961 >> 172.21.15.16:0/233308961 conn(0x7f0418079730 msgr2=0x7f0418079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 -- 172.21.15.16:0/233308961 shutdown_connections 2024-09-21T02:58:57.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.252+0000 7f041fa85640 1 -- 172.21.15.16:0/233308961 wait complete. 2024-09-21T02:58:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 Processor -- start 2024-09-21T02:58:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 -- start start 2024-09-21T02:58:57.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0418105300 0x7f0418102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 0x7f0418103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0418113a70 con 0x7f0418105300 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f04181138f0 con 0x7f04181049b0 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041fa85640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0418113bf0 con 0x7f041810de10 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041d821640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041d821640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41532/0 (socket says 172.21.15.16:41532) 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041d821640 1 -- 172.21.15.16:0/292450340 learned_addr learned my addr 172.21.15.16:0/292450340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041e022640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 0x7f0418103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.253+0000 7f041d020640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0418105300 0x7f0418102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 -- 172.21.15.16:0/292450340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 msgr2=0x7f0418103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:57.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 0x7f0418103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 -- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0418105300 msgr2=0x7f0418102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0418105300 0x7f0418102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 -- 172.21.15.16:0/292450340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04181a8780 con 0x7f04181049b0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041e022640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 0x7f0418103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041d821640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0414032fd0 tx=0x7f0414004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041403f030 con 0x7f04181049b0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04181a8a10 con 0x7f04181049b0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.254+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0414004490 con 0x7f04181049b0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.255+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04181a8f40 con 0x7f04181049b0 2024-09-21T02:58:57.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.255+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0414035070 con 0x7f04181049b0 2024-09-21T02:58:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.256+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f03fc0052c0 con 0x7f04181049b0 2024-09-21T02:58:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.257+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f0414042030 con 0x7f04181049b0 2024-09-21T02:58:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.258+0000 7f040effd640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 0x7f04040b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.258+0000 7f041d020640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 0x7f04040b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:57.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.258+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f04141079f0 con 0x7f04181049b0 2024-09-21T02:58:57.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.259+0000 7f041d020640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 0x7f04040b8b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0418079fd0 tx=0x7f040800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:57.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.261+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f04140d23c0 con 0x7f04181049b0 2024-09-21T02:58:57.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.502+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f03fc005880 con 0x7f04181049b0 2024-09-21T02:58:57.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.507+0000 7f040effd640 1 -- 172.21.15.16:0/292450340 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v21) ==== 74+0+296454 (secure 0 0 0) 0x7f04140d7300 con 0x7f04181049b0 2024-09-21T02:58:57.509 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 msgr2=0x7f04040b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 0x7f04040b8b80 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f0418079fd0 tx=0x7f040800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 msgr2=0x7f04181026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f0414032fd0 tx=0x7f0414004270 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 shutdown_connections 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f04040b6670 0x7f04040b8b80 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0418105300 0x7f0418102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f041810de10 0x7f0418103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 --2- 172.21.15.16:0/292450340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f04181049b0 0x7f04181026f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 >> 172.21.15.16:0/292450340 conn(0x7f0418079730 msgr2=0x7f04180ff730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 shutdown_connections 2024-09-21T02:58:57.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:57.511+0000 7f041fa85640 1 -- 172.21.15.16:0/292450340 wait complete. 2024-09-21T02:58:57.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:57 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/292450340' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:58:58.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:57 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/292450340' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:58:58.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:57 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/292450340' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json"} : dispatch 2024-09-21T02:58:58.225 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":21,"flags":0,"active_gid":14226,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6800","nonce":3994309618},{"type":"v1","addr":"172.21.15.16:6801","nonce":3994309618}]},"active_addr":"172.21.15.16:6801/3994309618","active_change":"2024-09-21T02:51:53.053676+0000","active_mgr_features":4540719139924082687,"available":true,"standbys":[{"gid":24136,"name":"z","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24170,"name":"y","mgr_features":4540719139924082687,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.27.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:10.4.8","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.7.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.17","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"oauth2-proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.51.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:latest","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB metrics exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many times ssh connections can fail liveness checks before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often ssh connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_requests":{"name":"max_requests","type":"int","level":"advanced","flags":0,"default_value":"500","min":"","max":"","enum_allowed":[],"desc":"Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.16:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":5,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":2855103234}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3468954010}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1946590608}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":1874751702}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.16:0","nonce":3106788870}]}]} 2024-09-21T02:58:58.235 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-09-21T02:58:58.235 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-09-21T02:58:58.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:58:58.494 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:58:58.867 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:58 smithi016 ceph-mon[31253]: pgmap v277: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:58:58.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 -- 172.21.15.16:0/196724688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 msgr2=0x7fba24110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 --2- 172.21.15.16:0/196724688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba24110670 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fba1c009900 tx=0x7fba1c02a410 comp rx=0 tx=0).stop 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 -- 172.21.15.16:0/196724688 shutdown_connections 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 --2- 172.21.15.16:0/196724688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba24110670 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 --2- 172.21.15.16:0/196724688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 0x7fba24109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 --2- 172.21.15.16:0/196724688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 0x7fba24103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 -- 172.21.15.16:0/196724688 >> 172.21.15.16:0/196724688 conn(0x7fba240ff600 msgr2=0x7fba24101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 -- 172.21.15.16:0/196724688 shutdown_connections 2024-09-21T02:58:58.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.959+0000 7fba2b841640 1 -- 172.21.15.16:0/196724688 wait complete. 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 Processor -- start 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 -- start start 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 0x7fba2419a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 0x7fba2419aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fba24113b40 con 0x7fba24109e40 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fba241139c0 con 0x7fba24103bb0 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba2b841640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fba24113cc0 con 0x7fba241094f0 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba29dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba29dde640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37602/0 (socket says 172.21.15.16:37602) 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba29dde640 1 -- 172.21.15.16:0/3230571461 learned_addr learned my addr 172.21.15.16:0/3230571461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba295dd640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 0x7fba2419a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.960+0000 7fba28ddc640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 0x7fba2419aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:58.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 -- 172.21.15.16:0/3230571461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 msgr2=0x7fba2419aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 0x7fba2419aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 -- 172.21.15.16:0/3230571461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 msgr2=0x7fba2419a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 0x7fba2419a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 -- 172.21.15.16:0/3230571461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba241a0420 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba29dde640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba1c004770 tx=0x7fba1c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c03f030 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fba1c003aa0 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba241a06b0 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba1c035070 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.961+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba241a6e70 con 0x7fba24109e40 2024-09-21T02:58:58.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.962+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fba24103fc0 con 0x7fba24109e40 2024-09-21T02:58:58.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.964+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fba1c042030 con 0x7fba24109e40 2024-09-21T02:58:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.964+0000 7fba1a7fc640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 0x7fba100b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:58:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.965+0000 7fba295dd640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 0x7fba100b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:58:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.965+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fba1c107fe0 con 0x7fba24109e40 2024-09-21T02:58:58.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.965+0000 7fba295dd640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 0x7fba100b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba20005fd0 tx=0x7fba20009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:58:58.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:58.968+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fba1c0ce630 con 0x7fba24109e40 2024-09-21T02:58:59.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.156+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fba2419b5e0 con 0x7fba24109e40 2024-09-21T02:58:59.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.158+0000 7fba1a7fc640 1 -- 172.21.15.16:0/3230571461 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23079 (secure 0 0 0) 0x7fba2419b5e0 con 0x7fba24109e40 2024-09-21T02:58:59.160 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:58:59.160 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:54.732993+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 msgr2=0x7fba100b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 0x7fba100b8b80 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fba20005fd0 tx=0x7fba20009680 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 msgr2=0x7fba2419fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fba1c004770 tx=0x7fba1c004270 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 shutdown_connections 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fba100b6670 0x7fba100b8b80 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fba24109e40 0x7fba2419fcf0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fba241094f0 0x7fba2419aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 --2- 172.21.15.16:0/3230571461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fba24103bb0 0x7fba2419a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:58:59.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 >> 172.21.15.16:0/3230571461 conn(0x7fba240ff600 msgr2=0x7fba24101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:58:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 shutdown_connections 2024-09-21T02:58:59.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:58:59.160+0000 7fba2b841640 1 -- 172.21.15.16:0/3230571461 wait complete. 2024-09-21T02:58:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:58 smithi018 ceph-mon[34587]: pgmap v277: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:58:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:58 smithi123 ceph-mon[34645]: pgmap v277: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 5.4 KiB/s wr, 5 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:58:59.792 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-09-21T02:58:59.793 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:59:00.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:58:59 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3230571461' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:00.052 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:00.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:58:59 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3230571461' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:00.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:58:59 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3230571461' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1851916989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c108670 msgr2=0x7fef9c108a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1851916989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c108670 0x7fef9c108a80 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fef84009900 tx=0x7fef8402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1851916989 shutdown_connections 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1851916989 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c108670 0x7fef9c108a80 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1851916989 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108fc0 0x7fef9c10f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1851916989 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c103230 0x7fef9c103640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1851916989 >> 172.21.15.16:0/1851916989 conn(0x7fef9c079ea0 msgr2=0x7fef9c07a2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:00.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1851916989 shutdown_connections 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.498+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1851916989 wait complete. 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.499+0000 7fefa1ddc640 1 Processor -- start 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.499+0000 7fefa1ddc640 1 -- start start 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108670 0x7fef9c19ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c108fc0 0x7fef9c19ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c112c80 con 0x7fef9c103230 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c112b00 con 0x7fef9c108670 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fefa1ddc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fef9c112e00 con 0x7fef9c108fc0 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37626/0 (socket says 172.21.15.16:37626) 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 -- 172.21.15.16:0/1375656904 learned_addr learned my addr 172.21.15.16:0/1375656904 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9affd640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108670 0x7fef9c19ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9bfff640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c108fc0 0x7fef9c19ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:00.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 -- 172.21.15.16:0/1375656904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c108fc0 msgr2=0x7fef9c19ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c108fc0 0x7fef9c19ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 -- 172.21.15.16:0/1375656904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108670 msgr2=0x7fef9c19ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108670 0x7fef9c19ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 -- 172.21.15.16:0/1375656904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef9c1a06f0 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.500+0000 7fef9b7fe640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c890 tx=0x7fef8c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.501+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c004220 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.501+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef8c0043c0 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.501+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef9c1a09e0 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.501+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef8c004a00 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.501+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef9c1a7140 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.502+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef9c103640 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.503+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fef8c0079b0 con 0x7fef9c103230 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.504+0000 7fef98ff9640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 0x7fef880b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.504+0000 7fef9affd640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 0x7fef880b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:00.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.504+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fef8c0e76b0 con 0x7fef9c103230 2024-09-21T02:59:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.505+0000 7fef9affd640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 0x7fef880b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef84009230 tx=0x7fef84031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:00.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.507+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fef8c014030 con 0x7fef9c103230 2024-09-21T02:59:00.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.696+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fef9c19b8b0 con 0x7fef9c103230 2024-09-21T02:59:00.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.698+0000 7fef98ff9640 1 -- 172.21.15.16:0/1375656904 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23079 (secure 0 0 0) 0x7fef8c0b2100 con 0x7fef9c103230 2024-09-21T02:59:00.700 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:00.700 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:54.732993+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 msgr2=0x7fef880b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 0x7fef880b8b80 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fef84009230 tx=0x7fef84031400 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 msgr2=0x7fef9c19a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fef8c00c890 tx=0x7fef8c00cd50 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 shutdown_connections 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fef880b6670 0x7fef880b8b80 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fef9c108fc0 0x7fef9c19ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fef9c108670 0x7fef9c19ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 --2- 172.21.15.16:0/1375656904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fef9c103230 0x7fef9c19a780 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 >> 172.21.15.16:0/1375656904 conn(0x7fef9c079ea0 msgr2=0x7fef9c101200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 shutdown_connections 2024-09-21T02:59:00.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:00.700+0000 7fefa1ddc640 1 -- 172.21.15.16:0/1375656904 wait complete. 2024-09-21T02:59:01.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:00 smithi016 ceph-mon[31253]: pgmap v278: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 1 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:59:01.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:00 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1375656904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:00 smithi018 ceph-mon[34587]: pgmap v278: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 1 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:59:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:00 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1375656904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:00 smithi123 ceph-mon[34645]: pgmap v278: 97 pgs: 1 activating+degraded, 1 activating, 1 peering, 94 active+clean; 642 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 1.1 KiB/s wr, 1 op/s; 1/78 objects degraded (1.282%); 0 B/s, 0 objects/s recovering 2024-09-21T02:59:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:00 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1375656904' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:01.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.0 flush_pg_stats 2024-09-21T02:59:01.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.1 flush_pg_stats 2024-09-21T02:59:01.404 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.2 flush_pg_stats 2024-09-21T02:59:01.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.3 flush_pg_stats 2024-09-21T02:59:01.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.4 flush_pg_stats 2024-09-21T02:59:01.405 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.5 flush_pg_stats 2024-09-21T02:59:01.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.6 flush_pg_stats 2024-09-21T02:59:01.406 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.7 flush_pg_stats 2024-09-21T02:59:01.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.8 flush_pg_stats 2024-09-21T02:59:01.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.9 flush_pg_stats 2024-09-21T02:59:01.407 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.10 flush_pg_stats 2024-09-21T02:59:01.408 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell osd.11 flush_pg_stats 2024-09-21T02:59:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:01 smithi018 ceph-mon[34587]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded) 2024-09-21T02:59:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:01 smithi018 ceph-mon[34587]: Cluster is now healthy 2024-09-21T02:59:02.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:01 smithi123 ceph-mon[34645]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded) 2024-09-21T02:59:02.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:01 smithi123 ceph-mon[34645]: Cluster is now healthy 2024-09-21T02:59:02.243 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.244 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.248 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.249 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.252 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.253 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.255 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.264 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:01 smithi016 ceph-mon[31253]: Health check cleared: PG_DEGRADED (was: Degraded data redundancy: 1/78 objects degraded (1.282%), 1 pg degraded) 2024-09-21T02:59:02.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:01 smithi016 ceph-mon[31253]: Cluster is now healthy 2024-09-21T02:59:02.271 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.281 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 -- 172.21.15.16:0/2453593512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c1083b0 msgr2=0x7f778c1087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 --2- 172.21.15.16:0/2453593512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c1083b0 0x7f778c1087c0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f7780009900 tx=0x7f778002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 -- 172.21.15.16:0/2453593512 shutdown_connections 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 --2- 172.21.15.16:0/2453593512 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c1083b0 0x7f778c1087c0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 --2- 172.21.15.16:0/2453593512 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f778c108d00 0x7f778c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 --2- 172.21.15.16:0/2453593512 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f778c102ac0 0x7f778c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 -- 172.21.15.16:0/2453593512 >> 172.21.15.16:0/2453593512 conn(0x7f778c079720 msgr2=0x7f778c079b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 -- 172.21.15.16:0/2453593512 shutdown_connections 2024-09-21T02:59:02.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.853+0000 7f7791f13640 1 -- 172.21.15.16:0/2453593512 wait complete. 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 Processor -- start 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 -- start start 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f778c102ac0 0x7f778c19b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f778c1083b0 0x7f778c19b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 0x7f778c19bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f778c1129c0 con 0x7f778c108d00 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f778c112840 con 0x7f778c1083b0 2024-09-21T02:59:02.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.854+0000 7f7791f13640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f778c112b40 con 0x7f778c102ac0 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 0x7f778c19bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 0x7f778c19bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37654/0 (socket says 172.21.15.16:37654) 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 -- 172.21.15.16:0/1271157025 learned_addr learned my addr 172.21.15.16:0/1271157025 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778affd640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f778c1083b0 0x7f778c19b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778b7fe640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f778c102ac0 0x7f778c19b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 -- 172.21.15.16:0/1271157025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f778c102ac0 msgr2=0x7f778c19b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f778c102ac0 0x7f778c19b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 -- 172.21.15.16:0/1271157025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f778c1083b0 msgr2=0x7f778c19b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f778c1083b0 0x7f778c19b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f778c1a6540 con 0x7f778c108d00 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f778bfff640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 0x7f778c19bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f778400d750 tx=0x7f778400dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784017070 con 0x7f778c108d00 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77840054d0 con 0x7f778c108d00 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f778c1a6830 con 0x7f778c108d00 2024-09-21T02:59:02.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7784004c60 con 0x7f778c108d00 2024-09-21T02:59:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.855+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f778c1a6d40 con 0x7f778c108d00 2024-09-21T02:59:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.856+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f778c063b50 con 0x7f778c108d00 2024-09-21T02:59:02.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.858+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7784009bd0 con 0x7f778c108d00 2024-09-21T02:59:02.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f7788ff9640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77740b6620 0x7f77740b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f778b7fe640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77740b6620 0x7f77740b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f77840e72d0 con 0x7f778c108d00 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f7788ff9640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] conn(0x7f77740c0410 0x7f77740c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f7784021e50 con 0x7f77740c0410 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f77840e7720 con 0x7f778c108d00 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f778affd640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] conn(0x7f77740c0410 0x7f77740c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.859+0000 7f778b7fe640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77740b6620 0x7f77740b8b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f778c06c380 tx=0x7f777c009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:02.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.860+0000 7f778affd640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] conn(0x7f77740c0410 0x7f77740c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:02.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.860+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== osd.6 v2:172.21.15.16:6818/3190074295 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f7784021e50 con 0x7f77740c0410 2024-09-21T02:59:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.874+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 --> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f778c063fd0 con 0x7f77740c0410 2024-09-21T02:59:02.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.874+0000 7f7788ff9640 1 -- 172.21.15.16:0/1271157025 <== osd.6 v2:172.21.15.16:6818/3190074295 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f778c063fd0 con 0x7f77740c0410 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] conn(0x7f77740c0410 msgr2=0x7f77740c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] conn(0x7f77740c0410 0x7f77740c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77740b6620 msgr2=0x7f77740b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f77740b6620 0x7f77740b8b30 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f778c06c380 tx=0x7f777c009680 comp rx=0 tx=0).stop 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 msgr2=0x7f778c19bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 --2- 172.21.15.16:0/1271157025 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f778c108d00 0x7f778c19bd30 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f778400d750 tx=0x7f778400dc10 comp rx=0 tx=0).stop 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f778bfff640 1 -- 172.21.15.16:0/1271157025 reap_dead start 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 shutdown_connections 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 >> 172.21.15.16:0/1271157025 conn(0x7f778c079720 msgr2=0x7f778c1064a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 shutdown_connections 2024-09-21T02:59:02.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:02.875+0000 7f7791f13640 1 -- 172.21.15.16:0/1271157025 wait complete. 2024-09-21T02:59:02.960 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:02 smithi016 ceph-mon[31253]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:02 smithi018 ceph-mon[34587]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:03.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:02 smithi123 ceph-mon[34645]: pgmap v279: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.6 KiB/s rd, 1.4 KiB/s wr, 3 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 -- 172.21.15.16:0/1735383183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8108cf0 msgr2=0x7f2ad810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 --2- 172.21.15.16:0/1735383183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8108cf0 0x7f2ad810f520 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f2ad0009ca0 tx=0x7f2ad002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 -- 172.21.15.16:0/1735383183 shutdown_connections 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 --2- 172.21.15.16:0/1735383183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad8108cf0 0x7f2ad810f520 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 --2- 172.21.15.16:0/1735383183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ad81083a0 0x7f2ad81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 --2- 172.21.15.16:0/1735383183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad8102a60 0x7f2ad8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 -- 172.21.15.16:0/1735383183 >> 172.21.15.16:0/1735383183 conn(0x7f2ad8079730 msgr2=0x7f2ad8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 -- 172.21.15.16:0/1735383183 shutdown_connections 2024-09-21T02:59:04.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.262+0000 7f2adef98640 1 -- 172.21.15.16:0/1735383183 wait complete. 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 Processor -- start 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 -- start start 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ad8102a60 0x7f2ad819a450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 0x7f2ad819a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad8108cf0 0x7f2ad819fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8113a80 con 0x7f2ad81083a0 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8113900 con 0x7f2ad8102a60 2024-09-21T02:59:04.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2adef98640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2ad8113c00 con 0x7f2ad8108cf0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2ad7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 0x7f2ad819a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2ad7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 0x7f2ad819a9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37662/0 (socket says 172.21.15.16:37662) 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2ad7fff640 1 -- 172.21.15.16:0/3364041912 learned_addr learned my addr 172.21.15.16:0/3364041912 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.263+0000 7f2add535640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad8108cf0 0x7f2ad819fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2adcd34640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ad8102a60 0x7f2ad819a450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 -- 172.21.15.16:0/3364041912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad8108cf0 msgr2=0x7f2ad819fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2ad8108cf0 0x7f2ad819fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 -- 172.21.15.16:0/3364041912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ad8102a60 msgr2=0x7f2ad819a450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2ad8102a60 0x7f2ad819a450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ad81a03c0 con 0x7f2ad81083a0 2024-09-21T02:59:04.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad7fff640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 0x7f2ad819a9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc00dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc017070 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2acc009c30 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ad81a06b0 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2acc003d30 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.264+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ad81a6e10 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.265+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f2ad8063b10 con 0x7f2ad81083a0 2024-09-21T02:59:04.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.267+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2acc003ed0 con 0x7f2ad81083a0 2024-09-21T02:59:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2ad5ffb640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ac00b65f0 0x7f2ac00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2adcd34640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ac00b65f0 0x7f2ac00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f2acc0e7590 con 0x7f2ad81083a0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2ad5ffb640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] conn(0x7f2ac00c03e0 0x7f2ac00c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f2acc021e70 con 0x7f2ac00c03e0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f2acc0e79e0 con 0x7f2ad81083a0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2add535640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] conn(0x7f2ac00c03e0 0x7f2ac00c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.268+0000 7f2adcd34640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ac00b65f0 0x7f2ac00b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80058c0 tx=0x7f2ac800a4c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.269+0000 7f2add535640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] conn(0x7f2ac00c03e0 0x7f2ac00c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.269+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== osd.3 v2:172.21.15.16:6810/3760097578 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f2acc021e70 con 0x7f2ac00c03e0 2024-09-21T02:59:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 --> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2ad8063f10 con 0x7f2ac00c03e0 2024-09-21T02:59:04.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2ad5ffb640 1 -- 172.21.15.16:0/3364041912 <== osd.3 v2:172.21.15.16:6810/3760097578 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2ad8063f10 con 0x7f2ac00c03e0 2024-09-21T02:59:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] conn(0x7f2ac00c03e0 msgr2=0x7f2ac00c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] conn(0x7f2ac00c03e0 0x7f2ac00c28d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ac00b65f0 msgr2=0x7f2ac00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ac00b65f0 0x7f2ac00b8b00 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f2ac80058c0 tx=0x7f2ac800a4c0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.283+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 msgr2=0x7f2ad819a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2adef98640 1 --2- 172.21.15.16:0/3364041912 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2ad81083a0 0x7f2ad819a9f0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f2acc005fd0 tx=0x7f2acc00dde0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2add535640 1 -- 172.21.15.16:0/3364041912 reap_dead start 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 shutdown_connections 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 >> 172.21.15.16:0/3364041912 conn(0x7f2ad8079730 msgr2=0x7f2ad8104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 shutdown_connections 2024-09-21T02:59:04.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.284+0000 7f2adef98640 1 -- 172.21.15.16:0/3364041912 wait complete. 2024-09-21T02:59:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 -- 172.21.15.16:0/3923370302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 msgr2=0x7f9a001098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 --2- 172.21.15.16:0/3923370302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a001098b0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f99e8009900 tx=0x7f99e802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:04.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 -- 172.21.15.16:0/3923370302 shutdown_connections 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 --2- 172.21.15.16:0/3923370302 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a001098b0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 --2- 172.21.15.16:0/3923370302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00109df0 0x7f9a00110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 --2- 172.21.15.16:0/3923370302 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a00103b60 0x7f9a00103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.919+0000 7f9a05986640 1 -- 172.21.15.16:0/3923370302 >> 172.21.15.16:0/3923370302 conn(0x7f9a0007b850 msgr2=0x7f9a0007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.920+0000 7f9a05986640 1 -- 172.21.15.16:0/3923370302 shutdown_connections 2024-09-21T02:59:04.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.920+0000 7f9a05986640 1 -- 172.21.15.16:0/3923370302 wait complete. 2024-09-21T02:59:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.920+0000 7f9a05986640 1 Processor -- start 2024-09-21T02:59:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 -- start start 2024-09-21T02:59:04.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a00103b60 0x7f9a0019a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a0019ab70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00109df0 0x7f9a0019fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113b00 con 0x7f9a001094a0 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113980 con 0x7f9a00103b60 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f9a05986640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9a00113c80 con 0x7f9a00109df0 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f99f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a0019ab70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f99f67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a0019ab70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49254/0 (socket says 172.21.15.16:49254) 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f99f67fc640 1 -- 172.21.15.16:0/2288940435 learned_addr learned my addr 172.21.15.16:0/2288940435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f99feffd640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a00103b60 0x7f9a0019a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.921+0000 7f99ff7fe640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00109df0 0x7f9a0019fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 -- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00109df0 msgr2=0x7f9a0019fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a00109df0 0x7f9a0019fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 -- 172.21.15.16:0/2288940435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a00103b60 msgr2=0x7f9a0019a5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9a00103b60 0x7f9a0019a5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a001a0540 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f99f67fc640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a0019ab70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8003cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e803f070 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a001a07d0 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f99e8031d60 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99e8035030 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.922+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a001a6f90 con 0x7f9a001094a0 2024-09-21T02:59:04.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.923+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9a00063ad0 con 0x7f9a001094a0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.925+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f99e8042070 con 0x7f9a001094a0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f9a04984640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99ec0b65f0 0x7f99ec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f99feffd640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99ec0b65f0 0x7f99ec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f99e8108140 con 0x7f9a001094a0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f9a04984640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] conn(0x7f99ec0c03e0 0x7f99ec0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f99ec0c2fa0 con 0x7f99ec0c03e0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f99e81084f0 con 0x7f9a001094a0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f99feffd640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99ec0b65f0 0x7f99ec0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005d00 tx=0x7f99f000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.926+0000 7f99ff7fe640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] conn(0x7f99ec0c03e0 0x7f99ec0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.927+0000 7f99ff7fe640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] conn(0x7f99ec0c03e0 0x7f99ec0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:04.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.928+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== osd.8 v2:172.21.15.123:6816/1022177743 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f99ec0c2fa0 con 0x7f99ec0c03e0 2024-09-21T02:59:04.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.941+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 --> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9a00063f00 con 0x7f99ec0c03e0 2024-09-21T02:59:04.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a04984640 1 -- 172.21.15.16:0/2288940435 <== osd.8 v2:172.21.15.123:6816/1022177743 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9a00063f00 con 0x7f99ec0c03e0 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] conn(0x7f99ec0c03e0 msgr2=0x7f99ec0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] conn(0x7f99ec0c03e0 0x7f99ec0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99ec0b65f0 msgr2=0x7f99ec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99ec0b65f0 0x7f99ec0b8b00 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f99f0005d00 tx=0x7f99f000b410 comp rx=0 tx=0).stop 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 msgr2=0x7f9a0019ab70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 --2- 172.21.15.16:0/2288940435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9a001094a0 0x7f9a0019ab70 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f99e8005ab0 tx=0x7f99e8003cd0 comp rx=0 tx=0).stop 2024-09-21T02:59:04.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f99ff7fe640 1 -- 172.21.15.16:0/2288940435 reap_dead start 2024-09-21T02:59:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 shutdown_connections 2024-09-21T02:59:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 >> 172.21.15.16:0/2288940435 conn(0x7f9a0007b850 msgr2=0x7f9a00107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 shutdown_connections 2024-09-21T02:59:04.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.942+0000 7f9a05986640 1 -- 172.21.15.16:0/2288940435 wait complete. 2024-09-21T02:59:05.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:04 smithi016 ceph-mon[31253]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:04 smithi018 ceph-mon[34587]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:04 smithi123 ceph-mon[34645]: pgmap v280: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 2.2 KiB/s rd, 1.2 KiB/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:05.770 INFO:teuthology.orchestra.run.smithi016.stdout:111669149756 2024-09-21T02:59:05.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.3 2024-09-21T02:59:05.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- 172.21.15.16:0/1087992322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102f60 msgr2=0x7f75c810fc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 --2- 172.21.15.16:0/1087992322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102f60 0x7f75c810fc80 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f75b8009900 tx=0x7f75b802a480 comp rx=0 tx=0).stop 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- 172.21.15.16:0/1087992322 shutdown_connections 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 --2- 172.21.15.16:0/1087992322 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102f60 0x7f75c810fc80 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 --2- 172.21.15.16:0/1087992322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c81101c0 0x7f75c81125e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 --2- 172.21.15.16:0/1087992322 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c8102610 0x7f75c8102a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- 172.21.15.16:0/1087992322 >> 172.21.15.16:0/1087992322 conn(0x7f75c8079750 msgr2=0x7f75c8079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- 172.21.15.16:0/1087992322 shutdown_connections 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- 172.21.15.16:0/1087992322 wait complete. 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 Processor -- start 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.111+0000 7f75d01bd640 1 -- start start 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 0x7f75c819c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c8102f60 0x7f75c819cd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c81101c0 0x7f75c81a1fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8115cf0 con 0x7f75c8102610 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8115b70 con 0x7f75c81101c0 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75d01bd640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75c8115e70 con 0x7f75c8102f60 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 0x7f75c819c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 0x7f75c819c760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49276/0 (socket says 172.21.15.16:49276) 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 -- 172.21.15.16:0/3171619471 learned_addr learned my addr 172.21.15.16:0/3171619471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75ce75a640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c81101c0 0x7f75c81a1fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cd758640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c8102f60 0x7f75c819cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 -- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c8102f60 msgr2=0x7f75c819cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75c8102f60 0x7f75c819cd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 -- 172.21.15.16:0/3171619471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c81101c0 msgr2=0x7f75c81a1fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c81101c0 0x7f75c81a1fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75c81a26d0 con 0x7f75c8102610 2024-09-21T02:59:05.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75ce75a640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75c81101c0 0x7f75c81a1fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.112+0000 7f75cdf59640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 0x7f75c819c760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c890 tx=0x7f75c400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.113+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004220 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.113+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c40043c0 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.113+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75c81a29c0 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.113+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c4004a00 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.113+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75c81a9120 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.114+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f75c810a7c0 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.116+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f75c40079b0 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.116+0000 7f75beffd640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75b40b65f0 0x7f75b40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.116+0000 7f75cd758640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75b40b65f0 0x7f75b40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.116+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f75c40e7830 con 0x7f75c8102610 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75beffd640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] conn(0x7f75b40c03e0 0x7f75b40c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f75b40c2fa0 con 0x7f75b40c03e0 2024-09-21T02:59:05.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f75c40e7c80 con 0x7f75c8102610 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75cd758640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75b40b65f0 0x7f75b40b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005f70 tx=0x7f75b8000bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75ce75a640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] conn(0x7f75b40c03e0 0x7f75b40c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.117+0000 7f75ce75a640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] conn(0x7f75b40c03e0 0x7f75b40c28d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.118+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== osd.11 v2:172.21.15.123:6824/4172042931 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f75b40c2fa0 con 0x7f75b40c03e0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.132+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 --> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f75c806c240 con 0x7f75b40c03e0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.132+0000 7f75beffd640 1 -- 172.21.15.16:0/3171619471 <== osd.11 v2:172.21.15.123:6824/4172042931 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f75c806c240 con 0x7f75b40c03e0 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] conn(0x7f75b40c03e0 msgr2=0x7f75b40c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] conn(0x7f75b40c03e0 0x7f75b40c28d0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75b40b65f0 msgr2=0x7f75b40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75b40b65f0 0x7f75b40b8b00 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f75b8005f70 tx=0x7f75b8000bc0 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 msgr2=0x7f75c819c760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 --2- 172.21.15.16:0/3171619471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75c8102610 0x7f75c819c760 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f75c400c890 tx=0x7f75c400cd50 comp rx=0 tx=0).stop 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75ce75a640 1 -- 172.21.15.16:0/3171619471 reap_dead start 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 shutdown_connections 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 >> 172.21.15.16:0/3171619471 conn(0x7f75c8079750 msgr2=0x7f75c8109d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 shutdown_connections 2024-09-21T02:59:05.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:05.133+0000 7f75d01bd640 1 -- 172.21.15.16:0/3171619471 wait complete. 2024-09-21T02:59:06.547 INFO:teuthology.orchestra.run.smithi016.stdout:296352743444 2024-09-21T02:59:06.548 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.11 2024-09-21T02:59:06.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 -- 172.21.15.16:0/2479550165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 msgr2=0x7fab0010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2479550165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0010f520 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7faaf8009ca0 tx=0x7faaf802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 -- 172.21.15.16:0/2479550165 shutdown_connections 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2479550165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0010f520 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2479550165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab001083a0 0x7fab001087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2479550165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab00102a60 0x7fab00102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 -- 172.21.15.16:0/2479550165 >> 172.21.15.16:0/2479550165 conn(0x7fab00079730 msgr2=0x7fab00079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 -- 172.21.15.16:0/2479550165 shutdown_connections 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.799+0000 7fab04b7a640 1 -- 172.21.15.16:0/2479550165 wait complete. 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 Processor -- start 2024-09-21T02:59:06.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 -- start start 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab00102a60 0x7fab0019a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab001083a0 0x7fab0019aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0019fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab00113ac0 con 0x7fab00108cf0 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab00113940 con 0x7fab001083a0 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7fab04b7a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab00113c40 con 0x7fab00102a60 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7faafed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0019fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7faafed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0019fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49298/0 (socket says 172.21.15.16:49298) 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.800+0000 7faafdd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab001083a0 0x7fab0019aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 -- 172.21.15.16:0/2173069332 learned_addr learned my addr 172.21.15.16:0/2173069332 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafe59c640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab00102a60 0x7fab0019a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 -- 172.21.15.16:0/2173069332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab00102a60 msgr2=0x7fab0019a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab00102a60 0x7fab0019a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 -- 172.21.15.16:0/2173069332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab001083a0 msgr2=0x7fab0019aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab001083a0 0x7fab0019aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab001a0460 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faafed9d640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0019fd30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005cc0 tx=0x7faaf802f470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf8003cf0 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab001a06f0 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faaf80044f0 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faaf802fc30 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.801+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab001a6eb0 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.802+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fab00063b50 con 0x7fab00108cf0 2024-09-21T02:59:06.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.804+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7faaf80046a0 con 0x7fab00108cf0 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faaf77fe640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7faaec0b65f0 0x7faaec0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faafe59c640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7faaec0b65f0 0x7faaec0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7faaf8108cb0 con 0x7fab00108cf0 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faaf77fe640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] conn(0x7faaec0c03e0 0x7faaec0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7faaf80020b0 con 0x7faaec0c03e0 2024-09-21T02:59:06.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7faaf8109100 con 0x7fab00108cf0 2024-09-21T02:59:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.805+0000 7faafdd9b640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] conn(0x7faaec0c03e0 0x7faaec0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.806+0000 7faafe59c640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7faaec0b65f0 0x7faaec0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae8009990 tx=0x7faae8009110 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.806+0000 7faafdd9b640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] conn(0x7faaec0c03e0 0x7faaec0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:06.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.807+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== osd.9 v2:172.21.15.16:6826/661150471 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7faaf80020b0 con 0x7faaec0c03e0 2024-09-21T02:59:06.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.820+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 --> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fab00063fd0 con 0x7faaec0c03e0 2024-09-21T02:59:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.820+0000 7faaf77fe640 1 -- 172.21.15.16:0/2173069332 <== osd.9 v2:172.21.15.16:6826/661150471 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fab00063fd0 con 0x7faaec0c03e0 2024-09-21T02:59:06.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] conn(0x7faaec0c03e0 msgr2=0x7faaec0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] conn(0x7faaec0c03e0 0x7faaec0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7faaec0b65f0 msgr2=0x7faaec0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7faaec0b65f0 0x7faaec0b8b00 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7faae8009990 tx=0x7faae8009110 comp rx=0 tx=0).stop 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 msgr2=0x7fab0019fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 --2- 172.21.15.16:0/2173069332 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab00108cf0 0x7fab0019fd30 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7faaf8005cc0 tx=0x7faaf802f470 comp rx=0 tx=0).stop 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7faafed9d640 1 -- 172.21.15.16:0/2173069332 reap_dead start 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 shutdown_connections 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 >> 172.21.15.16:0/2173069332 conn(0x7fab00079730 msgr2=0x7fab001052b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 shutdown_connections 2024-09-21T02:59:06.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:06.821+0000 7fab04b7a640 1 -- 172.21.15.16:0/2173069332 wait complete. 2024-09-21T02:59:07.135 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:06 smithi016 ceph-mon[31253]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 196 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:06 smithi018 ceph-mon[34587]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 196 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:06 smithi123 ceph-mon[34645]: pgmap v281: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 196 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:07.488 INFO:teuthology.orchestra.run.smithi016.stdout:253403070495 2024-09-21T02:59:07.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.9 2024-09-21T02:59:08.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 -- 172.21.15.16:0/3850878887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448010de60 msgr2=0x7f4480110330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 --2- 172.21.15.16:0/3850878887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448010de60 0x7f4480110330 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f4478009ca0 tx=0x7f447802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 -- 172.21.15.16:0/3850878887 shutdown_connections 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 --2- 172.21.15.16:0/3850878887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f448010de60 0x7f4480110330 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 --2- 172.21.15.16:0/3850878887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4480105380 0x7f448010d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 --2- 172.21.15.16:0/3850878887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4480104a30 0x7f4480104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 -- 172.21.15.16:0/3850878887 >> 172.21.15.16:0/3850878887 conn(0x7f4480079760 msgr2=0x7f4480079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 -- 172.21.15.16:0/3850878887 shutdown_connections 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.447+0000 7f448653c640 1 -- 172.21.15.16:0/3850878887 wait complete. 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 Processor -- start 2024-09-21T02:59:08.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 -- start start 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 0x7f448019a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4480105380 0x7f448019a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448010de60 0x7f448019fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4480113ac0 con 0x7f4480104a30 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.448+0000 7f448653c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4480113940 con 0x7f4480105380 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f448653c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4480113c40 con 0x7f448010de60 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 0x7f448019a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 0x7f448019a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49232/0 (socket says 172.21.15.16:49232) 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 -- 172.21.15.16:0/3461859644 learned_addr learned my addr 172.21.15.16:0/3461859644 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f4484ad9640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448010de60 0x7f448019fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447f7fe640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4480105380 0x7f448019a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 -- 172.21.15.16:0/3461859644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448010de60 msgr2=0x7f448019fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f448010de60 0x7f448019fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 -- 172.21.15.16:0/3461859644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4480105380 msgr2=0x7f448019a9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4480105380 0x7f448019a9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44801a02e0 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447ffff640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 0x7f448019a430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f447000c970 tx=0x7f447000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4470007c10 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44801a05d0 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4470007db0 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f447000f870 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.449+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44801a6df0 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.450+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4480101160 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.452+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f447000fa10 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447d7fa640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f44680b6620 0x7f44680b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447f7fe640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f44680b6620 0x7f44680b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f44700e7e00 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447d7fa640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] conn(0x7f44680c0410 0x7f44680c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f4470025ea0 con 0x7f44680c0410 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f44700e8250 con 0x7f4480104a30 2024-09-21T02:59:08.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f4484ad9640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] conn(0x7f44680c0410 0x7f44680c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.453+0000 7f447f7fe640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f44680b6620 0x7f44680b8b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f4474005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.454+0000 7f4484ad9640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] conn(0x7f44680c0410 0x7f44680c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.454+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== osd.0 v2:172.21.15.16:6802/1882705397 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f4470025ea0 con 0x7f44680c0410 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.468+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 --> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f448006b350 con 0x7f44680c0410 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.468+0000 7f447d7fa640 1 -- 172.21.15.16:0/3461859644 <== osd.0 v2:172.21.15.16:6802/1882705397 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f448006b350 con 0x7f44680c0410 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] conn(0x7f44680c0410 msgr2=0x7f44680c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] conn(0x7f44680c0410 0x7f44680c2900 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f44680b6620 msgr2=0x7f44680b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f44680b6620 0x7f44680b8b30 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f4474005fd0 tx=0x7f4474005850 comp rx=0 tx=0).stop 2024-09-21T02:59:08.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 msgr2=0x7f448019a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 --2- 172.21.15.16:0/3461859644 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4480104a30 0x7f448019a430 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f447000c970 tx=0x7f447000ce30 comp rx=0 tx=0).stop 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f4484ad9640 1 -- 172.21.15.16:0/3461859644 reap_dead start 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 shutdown_connections 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.469+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 >> 172.21.15.16:0/3461859644 conn(0x7f4480079760 msgr2=0x7f4480100780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.470+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 shutdown_connections 2024-09-21T02:59:08.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:04.470+0000 7f448653c640 1 -- 172.21.15.16:0/3461859644 wait complete. 2024-09-21T02:59:08.729 INFO:teuthology.orchestra.run.smithi016.stdout:38654705740 2024-09-21T02:59:08.730 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.0 2024-09-21T02:59:09.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:08 smithi016 ceph-mon[31253]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:09.150 INFO:teuthology.orchestra.run.smithi016.stdout:184683593773 2024-09-21T02:59:09.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.6 2024-09-21T02:59:09.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:08 smithi018 ceph-mon[34587]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:09.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:08 smithi123 ceph-mon[34645]: pgmap v282: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 -- 172.21.15.16:0/323315904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 msgr2=0x7f9504110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 --2- 172.21.15.16:0/323315904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504110650 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f94fc009ca0 tx=0x7f94fc02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 -- 172.21.15.16:0/323315904 shutdown_connections 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 --2- 172.21.15.16:0/323315904 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504110650 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 --2- 172.21.15.16:0/323315904 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f95041094d0 0x7f95041098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 --2- 172.21.15.16:0/323315904 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9504103b90 0x7f9504103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 -- 172.21.15.16:0/323315904 >> 172.21.15.16:0/323315904 conn(0x7f95040ff600 msgr2=0x7f9504101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 -- 172.21.15.16:0/323315904 shutdown_connections 2024-09-21T02:59:10.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.075+0000 7f950c258640 1 -- 172.21.15.16:0/323315904 wait complete. 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 Processor -- start 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 -- start start 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9504103b90 0x7f9504075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95041094d0 0x7f9504075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113ae0 con 0x7f9504109e20 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113960 con 0x7f9504103b90 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950c258640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9504113c60 con 0x7f95041094d0 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49336/0 (socket says 172.21.15.16:49336) 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 -- 172.21.15.16:0/26307820 learned_addr learned my addr 172.21.15.16:0/26307820 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:10.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f9509ff4640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9504103b90 0x7f9504075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f95097f3640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95041094d0 0x7f9504075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 -- 172.21.15.16:0/26307820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95041094d0 msgr2=0x7f9504075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f95041094d0 0x7f9504075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 -- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9504103b90 msgr2=0x7f9504075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.076+0000 7f950a7f5640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9504103b90 0x7f9504075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f950a7f5640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95041a86f0 con 0x7f9504109e20 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f950a7f5640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504076190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005cc0 tx=0x7f94fc02f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc003bf0 con 0x7f9504109e20 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94fc003d90 con 0x7f9504109e20 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95041a8980 con 0x7f9504109e20 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94fc02fbc0 con 0x7f9504109e20 2024-09-21T02:59:10.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.077+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95041a8ec0 con 0x7f9504109e20 2024-09-21T02:59:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.078+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9504063b50 con 0x7f9504109e20 2024-09-21T02:59:10.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.080+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f94fc02fd60 con 0x7f9504109e20 2024-09-21T02:59:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f94faffd640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f94f00b65f0 0x7f94f00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f9509ff4640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f94f00b65f0 0x7f94f00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f94fc108c70 con 0x7f9504109e20 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f94faffd640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] conn(0x7f94f00c03e0 0x7f94f00c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f94f00c2fa0 con 0x7f94f00c03e0 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f94fc1090c0 con 0x7f9504109e20 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f95097f3640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] conn(0x7f94f00c03e0 0x7f94f00c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.081+0000 7f9509ff4640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f94f00b65f0 0x7f94f00b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95000058c0 tx=0x7f950000a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.082+0000 7f95097f3640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] conn(0x7f94f00c03e0 0x7f94f00c28d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.082+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== osd.4 v2:172.21.15.18:6808/2331184101 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f94f00c2fa0 con 0x7f94f00c03e0 2024-09-21T02:59:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.097+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 --> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9504063fd0 con 0x7f94f00c03e0 2024-09-21T02:59:10.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f94faffd640 1 -- 172.21.15.16:0/26307820 <== osd.4 v2:172.21.15.18:6808/2331184101 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f9504063fd0 con 0x7f94f00c03e0 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] conn(0x7f94f00c03e0 msgr2=0x7f94f00c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] conn(0x7f94f00c03e0 0x7f94f00c28d0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f94f00b65f0 msgr2=0x7f94f00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f94f00b65f0 0x7f94f00b8b00 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f95000058c0 tx=0x7f950000a250 comp rx=0 tx=0).stop 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 msgr2=0x7f9504076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.098+0000 7f950c258640 1 --2- 172.21.15.16:0/26307820 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9504109e20 0x7f9504076190 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f94fc005cc0 tx=0x7f94fc02f380 comp rx=0 tx=0).stop 2024-09-21T02:59:10.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.099+0000 7f950a7f5640 1 -- 172.21.15.16:0/26307820 reap_dead start 2024-09-21T02:59:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.099+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 shutdown_connections 2024-09-21T02:59:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.099+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 >> 172.21.15.16:0/26307820 conn(0x7f95040ff600 msgr2=0x7f9504106a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.099+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 shutdown_connections 2024-09-21T02:59:10.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.099+0000 7f950c258640 1 -- 172.21.15.16:0/26307820 wait complete. 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 -- 172.21.15.16:0/1124504306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c1049c0 msgr2=0x7fdd9c104dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 --2- 172.21.15.16:0/1124504306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c1049c0 0x7fdd9c104dd0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009900 tx=0x7fdd8c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 -- 172.21.15.16:0/1124504306 shutdown_connections 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 --2- 172.21.15.16:0/1124504306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9c10de20 0x7fdd9c110310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 --2- 172.21.15.16:0/1124504306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9c105310 0x7fdd9c10d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 --2- 172.21.15.16:0/1124504306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c1049c0 0x7fdd9c104dd0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 -- 172.21.15.16:0/1124504306 >> 172.21.15.16:0/1124504306 conn(0x7fdd9c079730 msgr2=0x7fdd9c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 -- 172.21.15.16:0/1124504306 shutdown_connections 2024-09-21T02:59:10.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.533+0000 7fdda1d62640 1 -- 172.21.15.16:0/1124504306 wait complete. 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 Processor -- start 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 -- start start 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9c1049c0 0x7fdd9c102700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9c105310 0x7fdd9c102c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 0x7fdd9c103180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113a80 con 0x7fdd9c10de20 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113900 con 0x7fdd9c105310 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdda1d62640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdd9c113c00 con 0x7fdd9c1049c0 2024-09-21T02:59:10.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdd9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 0x7fdd9c103180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdd9bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 0x7fdd9c103180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49370/0 (socket says 172.21.15.16:49370) 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.534+0000 7fdd9bfff640 1 -- 172.21.15.16:0/4007972697 learned_addr learned my addr 172.21.15.16:0/4007972697 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9affd640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9c105310 0x7fdd9c102c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9b7fe640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9c1049c0 0x7fdd9c102700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 -- 172.21.15.16:0/4007972697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9c1049c0 msgr2=0x7fdd9c102700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd9c1049c0 0x7fdd9c102700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 -- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9c105310 msgr2=0x7fdd9c102c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdd9c105310 0x7fdd9c102c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd9c1a8700 con 0x7fdd9c10de20 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd9bfff640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 0x7fdd9c103180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009fd0 tx=0x7fdd9400def0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94017070 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd94003e20 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd9c1a89f0 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd94010460 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.535+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd9c1a8ee0 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.536+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fdd9c063b50 con 0x7fdd9c10de20 2024-09-21T02:59:10.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.538+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fdd94003fd0 con 0x7fdd9c10de20 2024-09-21T02:59:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd98ff9640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd840b65f0 0x7fdd840b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fdd940e76c0 con 0x7fdd9c10de20 2024-09-21T02:59:10.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd9b7fe640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd840b65f0 0x7fdd840b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd98ff9640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] conn(0x7fdd840c03e0 0x7fdd840c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fdd840c2fa0 con 0x7fdd840c03e0 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7fdd940e7b10 con 0x7fdd9c10de20 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.539+0000 7fdd9b7fe640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd840b65f0 0x7fdd840b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009230 tx=0x7fdd8c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.540+0000 7fdd9affd640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] conn(0x7fdd840c03e0 0x7fdd840c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:10.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.540+0000 7fdd9affd640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] conn(0x7fdd840c03e0 0x7fdd840c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:10.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.541+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== osd.10 v2:172.21.15.18:6824/1894161437 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fdd840c2fa0 con 0x7fdd840c03e0 2024-09-21T02:59:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 --> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fdd9c063f50 con 0x7fdd840c03e0 2024-09-21T02:59:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdd98ff9640 1 -- 172.21.15.16:0/4007972697 <== osd.10 v2:172.21.15.18:6824/1894161437 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fdd9c063f50 con 0x7fdd840c03e0 2024-09-21T02:59:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] conn(0x7fdd840c03e0 msgr2=0x7fdd840c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdda1d62640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] conn(0x7fdd840c03e0 0x7fdd840c28d0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd840b65f0 msgr2=0x7fdd840b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.555+0000 7fdda1d62640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fdd840b65f0 0x7fdd840b8b00 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fdd8c009230 tx=0x7fdd8c031400 comp rx=0 tx=0).stop 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 msgr2=0x7fdd9c103180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 --2- 172.21.15.16:0/4007972697 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdd9c10de20 0x7fdd9c103180 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fdd94009fd0 tx=0x7fdd9400def0 comp rx=0 tx=0).stop 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdd9bfff640 1 -- 172.21.15.16:0/4007972697 reap_dead start 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 shutdown_connections 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 >> 172.21.15.16:0/4007972697 conn(0x7fdd9c079730 msgr2=0x7fdd9c100870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 shutdown_connections 2024-09-21T02:59:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.556+0000 7fdda1d62640 1 -- 172.21.15.16:0/4007972697 wait complete. 2024-09-21T02:59:10.896 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:10 smithi016 ceph-mon[31253]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:10 smithi018 ceph-mon[34587]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:10 smithi123 ceph-mon[34645]: pgmap v283: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 1 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:11.327 INFO:teuthology.orchestra.run.smithi016.stdout:231928234020 2024-09-21T02:59:11.327 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.8 2024-09-21T02:59:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 -- 172.21.15.16:0/2786867351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49201049b0 msgr2=0x7f4920104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 --2- 172.21.15.16:0/2786867351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49201049b0 0x7f4920104dc0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f4914009900 tx=0x7f491402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:11.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 -- 172.21.15.16:0/2786867351 shutdown_connections 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 --2- 172.21.15.16:0/2786867351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f492010de10 0x7f4920110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 --2- 172.21.15.16:0/2786867351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4920105300 0x7f492010d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 --2- 172.21.15.16:0/2786867351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f49201049b0 0x7f4920104dc0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.506+0000 7f49287d3640 1 -- 172.21.15.16:0/2786867351 >> 172.21.15.16:0/2786867351 conn(0x7f4920079730 msgr2=0x7f4920079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.507+0000 7f49287d3640 1 -- 172.21.15.16:0/2786867351 shutdown_connections 2024-09-21T02:59:11.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.507+0000 7f49287d3640 1 -- 172.21.15.16:0/2786867351 wait complete. 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.507+0000 7f49287d3640 1 Processor -- start 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.507+0000 7f49287d3640 1 -- start start 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49201049b0 0x7f49201026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4920105300 0x7f4920102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 0x7f4920103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4920113a90 con 0x7f492010de10 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4920113910 con 0x7f49201049b0 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f49287d3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4920113c10 con 0x7f4920105300 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 0x7f4920103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 0x7f4920103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49414/0 (socket says 172.21.15.16:49414) 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 -- 172.21.15.16:0/334928243 learned_addr learned my addr 172.21.15.16:0/334928243 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:11.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f492656f640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49201049b0 0x7f49201026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4925d6e640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4920105300 0x7f4920102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 -- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4920105300 msgr2=0x7f4920102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4920105300 0x7f4920102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 -- 172.21.15.16:0/334928243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49201049b0 msgr2=0x7f49201026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49201049b0 0x7f49201026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.508+0000 7f4926d70640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49201a8780 con 0x7f492010de10 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f4926d70640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 0x7f4920103170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f491800eaa0 tx=0x7f491800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f491800cdb0 con 0x7f492010de10 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4918005020 con 0x7f492010de10 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49180116d0 con 0x7f492010de10 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49201a8a70 con 0x7f492010de10 2024-09-21T02:59:11.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.509+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49201a8f50 con 0x7f492010de10 2024-09-21T02:59:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.510+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4920063b50 con 0x7f492010de10 2024-09-21T02:59:11.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.512+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f49180051d0 con 0x7f492010de10 2024-09-21T02:59:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f49137fe640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f490c0b65f0 0x7f490c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f49180e7bb0 con 0x7f492010de10 2024-09-21T02:59:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f492656f640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f490c0b65f0 0x7f490c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f49137fe640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] conn(0x7f490c0c03e0 0x7f490c0c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f490c0c2fa0 con 0x7f490c0c03e0 2024-09-21T02:59:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.513+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7f49180e7f60 con 0x7f492010de10 2024-09-21T02:59:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.514+0000 7f4925d6e640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] conn(0x7f490c0c03e0 0x7f490c0c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.514+0000 7f492656f640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f490c0b65f0 0x7f490c0b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.514+0000 7f4925d6e640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] conn(0x7f490c0c03e0 0x7f490c0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.515+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== osd.5 v2:172.21.15.123:6808/2715140085 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f490c0c2fa0 con 0x7f490c0c03e0 2024-09-21T02:59:11.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.529+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 --> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f4920063fd0 con 0x7f490c0c03e0 2024-09-21T02:59:11.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49137fe640 1 -- 172.21.15.16:0/334928243 <== osd.5 v2:172.21.15.123:6808/2715140085 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f4920063fd0 con 0x7f490c0c03e0 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] conn(0x7f490c0c03e0 msgr2=0x7f490c0c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] conn(0x7f490c0c03e0 0x7f490c0c28d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f490c0b65f0 msgr2=0x7f490c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f490c0b65f0 0x7f490c0b8b00 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f4914009230 tx=0x7f4914031400 comp rx=0 tx=0).stop 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 msgr2=0x7f4920103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 --2- 172.21.15.16:0/334928243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f492010de10 0x7f4920103170 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f491800eaa0 tx=0x7f491800ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f4926d70640 1 -- 172.21.15.16:0/334928243 reap_dead start 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 shutdown_connections 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 >> 172.21.15.16:0/334928243 conn(0x7f4920079730 msgr2=0x7f4920100770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 shutdown_connections 2024-09-21T02:59:11.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:11.530+0000 7f49287d3640 1 -- 172.21.15.16:0/334928243 wait complete. 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 -- 172.21.15.16:0/1618463027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca410de30 msgr2=0x7efca4110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/1618463027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca410de30 0x7efca4110320 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7efc9c009ca0 tx=0x7efc9c02a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 -- 172.21.15.16:0/1618463027 shutdown_connections 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/1618463027 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca410de30 0x7efca4110320 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/1618463027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 0x7efca410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/1618463027 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca41049d0 0x7efca4104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 -- 172.21.15.16:0/1618463027 >> 172.21.15.16:0/1618463027 conn(0x7efca4079740 msgr2=0x7efca4079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 -- 172.21.15.16:0/1618463027 shutdown_connections 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 -- 172.21.15.16:0/1618463027 wait complete. 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.982+0000 7efcaaa6d640 1 Processor -- start 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 -- start start 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 0x7efca419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 0x7efca419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca410de30 0x7efca419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113a90 con 0x7efca41049d0 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113910 con 0x7efca410de30 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efcaaa6d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7efca4113c10 con 0x7efca4105320 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 0x7efca419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 0x7efca419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49318/0 (socket says 172.21.15.16:49318) 2024-09-21T02:59:11.611 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 -- 172.21.15.16:0/3624622785 learned_addr learned my addr 172.21.15.16:0/3624622785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca900a640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca410de30 0x7efca419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca3fff640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 0x7efca419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 -- 172.21.15.16:0/3624622785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 msgr2=0x7efca419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 0x7efca419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 -- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca410de30 msgr2=0x7efca419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca410de30 0x7efca419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca8809640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efca41a0390 con 0x7efca41049d0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca900a640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7efca410de30 0x7efca419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.983+0000 7efca3fff640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efca4105320 0x7efca419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efca8809640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 0x7efca419a4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c970 tx=0x7efc9400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc94007c10 con 0x7efca41049d0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efca41a0680 con 0x7efca41049d0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc94007db0 con 0x7efca41049d0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc9400f7d0 con 0x7efca41049d0 2024-09-21T02:59:11.612 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.984+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efca41a6ea0 con 0x7efca41049d0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.985+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7efca4108440 con 0x7efca41049d0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.987+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7efc94018480 con 0x7efca41049d0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca1ffb640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7efc900b6620 0x7efc900b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca3fff640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7efc900b6620 0x7efc900b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7efc940e7da0 con 0x7efca41049d0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca1ffb640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] conn(0x7efc900c0410 0x7efc900c2900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7efc900c2fd0 con 0x7efc900c0410 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7efc940e81f0 con 0x7efca41049d0 2024-09-21T02:59:11.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca3fff640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7efc900b6620 0x7efc900b8b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005e00 tx=0x7efc8c005c70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.988+0000 7efca900a640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] conn(0x7efc900c0410 0x7efc900c2900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.989+0000 7efca900a640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] conn(0x7efc900c0410 0x7efc900c2900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:09.990+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== osd.1 v2:172.21.15.18:6800/1950134733 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7efc900c2fd0 con 0x7efc900c0410 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.003+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 --> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7efca406bf20 con 0x7efc900c0410 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efca1ffb640 1 -- 172.21.15.16:0/3624622785 <== osd.1 v2:172.21.15.18:6800/1950134733 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7efca406bf20 con 0x7efc900c0410 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] conn(0x7efc900c0410 msgr2=0x7efc900c2900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] conn(0x7efc900c0410 0x7efc900c2900 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7efc900b6620 msgr2=0x7efc900b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7efc900b6620 0x7efc900b8b30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efc8c005e00 tx=0x7efc8c005c70 comp rx=0 tx=0).stop 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 msgr2=0x7efca419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efcaaa6d640 1 --2- 172.21.15.16:0/3624622785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7efca41049d0 0x7efca419a4e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc9400c970 tx=0x7efc9400ce30 comp rx=0 tx=0).stop 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.004+0000 7efca900a640 1 -- 172.21.15.16:0/3624622785 reap_dead start 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.005+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 shutdown_connections 2024-09-21T02:59:11.614 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.005+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 >> 172.21.15.16:0/3624622785 conn(0x7efca4079740 msgr2=0x7efca41079e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.005+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 shutdown_connections 2024-09-21T02:59:11.615 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.005+0000 7efcaaa6d640 1 -- 172.21.15.16:0/3624622785 wait complete. 2024-09-21T02:59:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 -- 172.21.15.16:0/2160959728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 msgr2=0x7febcc1087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 --2- 172.21.15.16:0/2160959728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc1087f0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7febbc009900 tx=0x7febbc02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 -- 172.21.15.16:0/2160959728 shutdown_connections 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 --2- 172.21.15.16:0/2160959728 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc1087f0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 --2- 172.21.15.16:0/2160959728 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febcc108d30 0x7febcc10f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 --2- 172.21.15.16:0/2160959728 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febcc102aa0 0x7febcc102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.287+0000 7febd40d7640 1 -- 172.21.15.16:0/2160959728 >> 172.21.15.16:0/2160959728 conn(0x7febcc079750 msgr2=0x7febcc079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.288+0000 7febd40d7640 1 -- 172.21.15.16:0/2160959728 shutdown_connections 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.288+0000 7febd40d7640 1 -- 172.21.15.16:0/2160959728 wait complete. 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.288+0000 7febd40d7640 1 Processor -- start 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.288+0000 7febd40d7640 1 -- start start 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febcc102aa0 0x7febcc19b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc19b7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febcc108d30 0x7febcc19bd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7febcc1129f0 con 0x7febcc1083e0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7febcc112870 con 0x7febcc102aa0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd40d7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7febcc112b70 con 0x7febcc108d30 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc19b7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc19b7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49356/0 (socket says 172.21.15.16:49356) 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 -- 172.21.15.16:0/4182131756 learned_addr learned my addr 172.21.15.16:0/4182131756 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1e73640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febcc102aa0 0x7febcc19b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd2674640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febcc108d30 0x7febcc19bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 -- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febcc108d30 msgr2=0x7febcc19bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7febcc108d30 0x7febcc19bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 -- 172.21.15.16:0/4182131756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febcc102aa0 msgr2=0x7febcc19b2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7febcc102aa0 0x7febcc19b2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7febcc1a6560 con 0x7febcc1083e0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.289+0000 7febd1672640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc19b7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7febbc006010 tx=0x7febbc004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.290+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc03f030 con 0x7febcc1083e0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.290+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7febbc003aa0 con 0x7febcc1083e0 2024-09-21T02:59:12.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.290+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7febbc035070 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.290+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7febcc1a67f0 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.290+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7febcc1a6cd0 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.291+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7febcc063b50 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.292+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7febbc042030 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.293+0000 7febc2ffd640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7febb80b65f0 0x7febb80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.293+0000 7febd1e73640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7febb80b65f0 0x7febb80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.293+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7febbc107ab0 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febc2ffd640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] conn(0x7febb80c03e0 0x7febb80c28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7febbc03ce50 con 0x7febb80c03e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7febbc107f00 con 0x7febcc1083e0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febd1e73640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7febb80b65f0 0x7febb80b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febcc06c6c0 tx=0x7febc800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febd2674640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] conn(0x7febb80c03e0 0x7febb80c28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.294+0000 7febd2674640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] conn(0x7febb80c03e0 0x7febb80c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.295+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== osd.2 v2:172.21.15.123:6800/3497511537 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7febbc03ce50 con 0x7febb80c03e0 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.309+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 --> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7febcc063f50 con 0x7febb80c03e0 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.309+0000 7febc2ffd640 1 -- 172.21.15.16:0/4182131756 <== osd.2 v2:172.21.15.123:6800/3497511537 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7febcc063f50 con 0x7febb80c03e0 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.309+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] conn(0x7febb80c03e0 msgr2=0x7febb80c28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.309+0000 7febd40d7640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] conn(0x7febb80c03e0 0x7febb80c28d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7febb80b65f0 msgr2=0x7febb80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7febb80b65f0 0x7febb80b8b00 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7febcc06c6c0 tx=0x7febc800b410 comp rx=0 tx=0).stop 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 msgr2=0x7febcc19b7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 --2- 172.21.15.16:0/4182131756 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7febcc1083e0 0x7febcc19b7e0 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7febbc006010 tx=0x7febbc004060 comp rx=0 tx=0).stop 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd2674640 1 -- 172.21.15.16:0/4182131756 reap_dead start 2024-09-21T02:59:12.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 shutdown_connections 2024-09-21T02:59:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 >> 172.21.15.16:0/4182131756 conn(0x7febcc079750 msgr2=0x7febcc106470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 shutdown_connections 2024-09-21T02:59:12.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.310+0000 7febd40d7640 1 -- 172.21.15.16:0/4182131756 wait complete. 2024-09-21T02:59:12.565 INFO:teuthology.orchestra.run.smithi016.stdout:81604378692 2024-09-21T02:59:12.566 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.2 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 -- 172.21.15.16:0/2877877478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 msgr2=0x7ff9f41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/2877877478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f41087b0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4009900 tx=0x7ff9e402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 -- 172.21.15.16:0/2877877478 shutdown_connections 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/2877877478 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f41087b0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/2877877478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9f4108cf0 0x7ff9f410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/2877877478 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f4102a60 0x7ff9f4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 -- 172.21.15.16:0/2877877478 >> 172.21.15.16:0/2877877478 conn(0x7ff9f4079730 msgr2=0x7ff9f4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 -- 172.21.15.16:0/2877877478 shutdown_connections 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.710+0000 7ff9fbadf640 1 -- 172.21.15.16:0/2877877478 wait complete. 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 Processor -- start 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 -- start start 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f4102a60 0x7ff9f419b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f419b810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9f4108cf0 0x7ff9f419bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4112960 con 0x7ff9f41083a0 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f41127e0 con 0x7ff9f4102a60 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fbadf640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9f4112ae0 con 0x7ff9f4108cf0 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9f907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f419b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9f907a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f419b810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49390/0 (socket says 172.21.15.16:49390) 2024-09-21T02:59:12.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9f907a640 1 -- 172.21.15.16:0/3378533474 learned_addr learned my addr 172.21.15.16:0/3378533474 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9f987b640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f4102a60 0x7ff9f419b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.711+0000 7ff9fa07c640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9f4108cf0 0x7ff9f419bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 -- 172.21.15.16:0/3378533474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9f4108cf0 msgr2=0x7ff9f419bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9f4108cf0 0x7ff9f419bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 -- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f4102a60 msgr2=0x7ff9f419b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9f4102a60 0x7ff9f419b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9f41a65e0 con 0x7ff9f41083a0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9f907a640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f419b810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4006010 tx=0x7ff9e4004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e403f030 con 0x7ff9f41083a0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9e4003aa0 con 0x7ff9f41083a0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9f41a6870 con 0x7ff9f41083a0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9e4035070 con 0x7ff9f41083a0 2024-09-21T02:59:12.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.712+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9f41a6da0 con 0x7ff9f41083a0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.713+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ff9f4063b50 con 0x7ff9f41083a0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.715+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff9e4042030 con 0x7ff9f41083a0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9eaffd640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9e00b6670 0x7ff9e00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9f987b640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9e00b6670 0x7ff9e00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7ff9e4107da0 con 0x7ff9f41083a0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9eaffd640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] conn(0x7ff9e00c0460 0x7ff9e00c2950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ff9e4032970 con 0x7ff9e00c0460 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_get_version_reply(handle=1 version=77) ==== 24+0+0 (secure 0 0 0) 0x7ff9e41081f0 con 0x7ff9f41083a0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9f987b640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9e00b6670 0x7ff9e00b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9f406ba20 tx=0x7ff9f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.716+0000 7ff9fa07c640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] conn(0x7ff9e00c0460 0x7ff9e00c2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.717+0000 7ff9fa07c640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] conn(0x7ff9e00c0460 0x7ff9e00c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.717+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== osd.7 v2:172.21.15.18:6816/3027010231 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ff9e4032970 con 0x7ff9e00c0460 2024-09-21T02:59:12.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.731+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 --> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ff9f4063f80 con 0x7ff9e00c0460 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9eaffd640 1 -- 172.21.15.16:0/3378533474 <== osd.7 v2:172.21.15.18:6816/3027010231 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ff9f4063f80 con 0x7ff9e00c0460 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] conn(0x7ff9e00c0460 msgr2=0x7ff9e00c2950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] conn(0x7ff9e00c0460 0x7ff9e00c2950 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9e00b6670 msgr2=0x7ff9e00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9e00b6670 0x7ff9e00b8b80 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff9f406ba20 tx=0x7ff9f0005f90 comp rx=0 tx=0).stop 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 msgr2=0x7ff9f419b810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 --2- 172.21.15.16:0/3378533474 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9f41083a0 0x7ff9f419b810 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff9e4006010 tx=0x7ff9e4004060 comp rx=0 tx=0).stop 2024-09-21T02:59:12.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fa07c640 1 -- 172.21.15.16:0/3378533474 reap_dead start 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 shutdown_connections 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.732+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 >> 172.21.15.16:0/3378533474 conn(0x7ff9f4079730 msgr2=0x7ff9f4104480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.733+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 shutdown_connections 2024-09-21T02:59:12.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:10.733+0000 7ff9fbadf640 1 -- 172.21.15.16:0/3378533474 wait complete. 2024-09-21T02:59:13.136 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:12 smithi016 ceph-mon[31253]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:13.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:12 smithi018 ceph-mon[34587]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:13.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:12 smithi123 ceph-mon[34645]: pgmap v284: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s rd, 170 B/s wr, 2 op/s; 0 B/s, 0 objects/s recovering 2024-09-21T02:59:13.435 INFO:teuthology.orchestra.run.smithi016.stdout:206158430249 2024-09-21T02:59:13.435 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.7 2024-09-21T02:59:14.067 INFO:teuthology.orchestra.run.smithi016.stdout:60129542215 2024-09-21T02:59:14.067 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.1 2024-09-21T02:59:14.588 INFO:teuthology.orchestra.run.smithi016.stdout:137438953527 2024-09-21T02:59:14.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.4 2024-09-21T02:59:15.054 INFO:teuthology.orchestra.run.smithi016.stdout:274877906971 2024-09-21T02:59:15.054 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.10 2024-09-21T02:59:15.174 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:14 smithi016 ceph-mon[31253]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T02:59:15.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:14 smithi018 ceph-mon[34587]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T02:59:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:14 smithi123 ceph-mon[34645]: pgmap v285: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T02:59:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:15.425 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:15.468 INFO:teuthology.orchestra.run.smithi016.stdout:163208757299 2024-09-21T02:59:15.469 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd last-stat-seq osd.5 2024-09-21T02:59:15.474 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:16.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 -- 172.21.15.16:0/4025370409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 msgr2=0x7f414c1029f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 --2- 172.21.15.16:0/4025370409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c1029f0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f4148009900 tx=0x7f414802a480 comp rx=0 tx=0).stop 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 -- 172.21.15.16:0/4025370409 shutdown_connections 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 --2- 172.21.15.16:0/4025370409 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f414c110190 0x7f414c1125b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 --2- 172.21.15.16:0/4025370409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 0x7f414c10fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 --2- 172.21.15.16:0/4025370409 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c1029f0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.401+0000 7f4153d04640 1 -- 172.21.15.16:0/4025370409 >> 172.21.15.16:0/4025370409 conn(0x7f414c079710 msgr2=0x7f414c079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.402+0000 7f4153d04640 1 -- 172.21.15.16:0/4025370409 shutdown_connections 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.402+0000 7f4153d04640 1 -- 172.21.15.16:0/4025370409 wait complete. 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.402+0000 7f4153d04640 1 Processor -- start 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.402+0000 7f4153d04640 1 -- start start 2024-09-21T02:59:16.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 0x7f414c19cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f414c110190 0x7f414c1a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f414c115c80 con 0x7f414c1025e0 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f414c115b00 con 0x7f414c110190 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f414c115e00 con 0x7f414c102f30 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40742/0 (socket says 172.21.15.16:40742) 2024-09-21T02:59:16.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 -- 172.21.15.16:0/3296686196 learned_addr learned my addr 172.21.15.16:0/3296686196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f415129f640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 0x7f414c19cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 -- 172.21.15.16:0/3296686196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 msgr2=0x7f414c19cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 0x7f414c19cc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 -- 172.21.15.16:0/3296686196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f414c110190 msgr2=0x7f414c1a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f414c110190 0x7f414c1a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 -- 172.21.15.16:0/3296686196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f414c1a2600 con 0x7f414c1025e0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4151aa0640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4148005940 tx=0x7f41480042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f414803f030 con 0x7f414c1025e0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f414c1a2890 con 0x7f414c1025e0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4148003cb0 con 0x7f414c1025e0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4148035070 con 0x7f414c1025e0 2024-09-21T02:59:16.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.403+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f414c1a9050 con 0x7f414c1025e0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.404+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f414c06c280 con 0x7f414c1025e0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.407+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4148042030 con 0x7f414c1025e0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.408+0000 7f4142ffd640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 0x7f41380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.408+0000 7f415129f640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 0x7f41380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.408+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f4148108230 con 0x7f414c1025e0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.408+0000 7f415129f640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 0x7f41380b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f413c005fd0 tx=0x7f413c005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:16.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.411+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f41480ce820 con 0x7f414c1025e0 2024-09-21T02:59:16.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.605+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7f414c19d7c0 con 0x7f414c1025e0 2024-09-21T02:59:16.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.605+0000 7f4142ffd640 1 -- 172.21.15.16:0/3296686196 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f41480d2c00 con 0x7f414c1025e0 2024-09-21T02:59:16.607 INFO:teuthology.orchestra.run.smithi016.stdout:206158430250 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.607+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 msgr2=0x7f41380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.607+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 0x7f41380b8b00 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f413c005fd0 tx=0x7f413c005850 comp rx=0 tx=0).stop 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.607+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 msgr2=0x7f414c19c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.607+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f4148005940 tx=0x7f41480042f0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 shutdown_connections 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f41380b65f0 0x7f41380b8b00 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.609 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f414c110190 0x7f414c1a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f414c102f30 0x7f414c19cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 --2- 172.21.15.16:0/3296686196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f414c1025e0 0x7f414c19c690 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 >> 172.21.15.16:0/3296686196 conn(0x7f414c079710 msgr2=0x7f414c108c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 shutdown_connections 2024-09-21T02:59:16.610 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.608+0000 7f4153d04640 1 -- 172.21.15.16:0/3296686196 wait complete. 2024-09-21T02:59:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 -- 172.21.15.16:0/3382557073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178107b30 msgr2=0x7fa178107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 --2- 172.21.15.16:0/3382557073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178107b30 0x7fa178107f40 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fa16800aa10 tx=0x7fa16802b720 comp rx=0 tx=0).stop 2024-09-21T02:59:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 -- 172.21.15.16:0/3382557073 shutdown_connections 2024-09-21T02:59:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 --2- 172.21.15.16:0/3382557073 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178107b30 0x7fa178107f40 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 --2- 172.21.15.16:0/3382557073 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178108480 0x7fa17810ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 --2- 172.21.15.16:0/3382557073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 0x7fa178102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 -- 172.21.15.16:0/3382557073 >> 172.21.15.16:0/3382557073 conn(0x7fa178077c60 msgr2=0x7fa1780780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 -- 172.21.15.16:0/3382557073 shutdown_connections 2024-09-21T02:59:16.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.620+0000 7fa17f78c640 1 -- 172.21.15.16:0/3382557073 wait complete. 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 Processor -- start 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 -- start start 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 0x7fa1780fede0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178107b30 0x7fa1780ff320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa1781118c0 con 0x7fa178108480 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa178111740 con 0x7fa178107b30 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17f78c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa178111a40 con 0x7fa1781021f0 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17dd29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17dd29640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40768/0 (socket says 172.21.15.16:40768) 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17dd29640 1 -- 172.21.15.16:0/1450829182 learned_addr learned my addr 172.21.15.16:0/1450829182 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17d528640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 0x7fa1780fede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.624 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.621+0000 7fa17cd27640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178107b30 0x7fa1780ff320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 -- 172.21.15.16:0/1450829182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 msgr2=0x7fa1780fede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 0x7fa1780fede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 -- 172.21.15.16:0/1450829182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178107b30 msgr2=0x7fa1780ff320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178107b30 0x7fa1780ff320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 -- 172.21.15.16:0/1450829182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1781a87f0 con 0x7fa178108480 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17dd29640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa17000eb10 tx=0x7fa17000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170004b20 con 0x7fa178108480 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1781a8ae0 con 0x7fa178108480 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa170004cc0 con 0x7fa178108480 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1781a8ff0 con 0x7fa178108480 2024-09-21T02:59:16.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.622+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa170010670 con 0x7fa178108480 2024-09-21T02:59:16.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.624+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa178102600 con 0x7fa178108480 2024-09-21T02:59:16.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.627+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fa170005000 con 0x7fa178108480 2024-09-21T02:59:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.628+0000 7fa16e7fc640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 0x7fa1640b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.628+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fa1700e7d20 con 0x7fa178108480 2024-09-21T02:59:16.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.628+0000 7fa17d528640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 0x7fa1640b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:16.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.628+0000 7fa17d528640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 0x7fa1640b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa178100830 tx=0x7fa174005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:16.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.632+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa1700ae400 con 0x7fa178108480 2024-09-21T02:59:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.823+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fa1781000e0 con 0x7fa178108480 2024-09-21T02:59:16.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.824+0000 7fa16e7fc640 1 -- 172.21.15.16:0/1450829182 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fa1781000e0 con 0x7fa178108480 2024-09-21T02:59:16.825 INFO:teuthology.orchestra.run.smithi016.stdout:231928234022 2024-09-21T02:59:16.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 msgr2=0x7fa1640b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 0x7fa1640b8b80 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa178100830 tx=0x7fa174005f90 comp rx=0 tx=0).stop 2024-09-21T02:59:16.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 msgr2=0x7fa1780ff860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fa17000eb10 tx=0x7fa17000c2d0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 shutdown_connections 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa1640b6670 0x7fa1640b8b80 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa178108480 0x7fa1780ff860 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa178107b30 0x7fa1780ff320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 --2- 172.21.15.16:0/1450829182 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa1781021f0 0x7fa1780fede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 >> 172.21.15.16:0/1450829182 conn(0x7fa178077c60 msgr2=0x7fa178103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 shutdown_connections 2024-09-21T02:59:16.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:16.828+0000 7fa17f78c640 1 -- 172.21.15.16:0/1450829182 wait complete. 2024-09-21T02:59:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:16 smithi018 ceph-mon[34587]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3296686196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:59:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1450829182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:59:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:16 smithi123 ceph-mon[34645]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3296686196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:59:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1450829182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:59:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:16 smithi016 ceph-mon[31253]: pgmap v286: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3296686196' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 7} : dispatch 2024-09-21T02:59:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1450829182' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 8} : dispatch 2024-09-21T02:59:17.725 INFO:tasks.cephadm.ceph_manager.ceph:need seq 206158430249 got 206158430250 for osd.7 2024-09-21T02:59:17.725 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 -- 172.21.15.16:0/824092992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 msgr2=0x7f01f0103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 --2- 172.21.15.16:0/824092992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f0103f70 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f01ec009900 tx=0x7f01ec02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 -- 172.21.15.16:0/824092992 shutdown_connections 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 --2- 172.21.15.16:0/824092992 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f0110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 --2- 172.21.15.16:0/824092992 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 0x7f01f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 --2- 172.21.15.16:0/824092992 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f0103f70 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 -- 172.21.15.16:0/824092992 >> 172.21.15.16:0/824092992 conn(0x7f01f007b850 msgr2=0x7f01f007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 -- 172.21.15.16:0/824092992 shutdown_connections 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.464+0000 7f01f8758640 1 -- 172.21.15.16:0/824092992 wait complete. 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 Processor -- start 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 -- start start 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 0x7f01f019aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f019fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0113ab0 con 0x7f01f0103b60 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0113930 con 0x7f01f01094a0 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f8758640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01f0113c30 con 0x7f01f0109df0 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f64f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f64f4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40794/0 (socket says 172.21.15.16:40794) 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f64f4640 1 -- 172.21.15.16:0/2539991527 learned_addr learned my addr 172.21.15.16:0/2539991527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f5cf3640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 0x7f01f019aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.465+0000 7f01f6cf5640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f019fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 -- 172.21.15.16:0/2539991527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 msgr2=0x7f01f019fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f019fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 -- 172.21.15.16:0/2539991527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 msgr2=0x7f01f019aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 0x7f01f019aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 -- 172.21.15.16:0/2539991527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01f01a03f0 con 0x7f01f0103b60 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f6cf5640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f64f4640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005940 tx=0x7f01ec004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec03f030 con 0x7f01f0103b60 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01ec003aa0 con 0x7f01f0103b60 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01f01a0680 con 0x7f01f0103b60 2024-09-21T02:59:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01ec035070 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.466+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01f01a6e40 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.467+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f01f0104de0 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.470+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f01ec042030 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.470+0000 7f01e77fe640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 0x7f01dc0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.470+0000 7f01f5cf3640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 0x7f01dc0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.470+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f01ec107f00 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.471+0000 7f01f5cf3640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 0x7f01dc0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01f0109200 tx=0x7f01e0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.474+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01ec0ce550 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.664+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f01f0103f70 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.664+0000 7f01e77fe640 1 -- 172.21.15.16:0/2539991527 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f01ec0d2930 con 0x7f01f0103b60 2024-09-21T02:59:17.875 INFO:teuthology.orchestra.run.smithi016.stdout:296352743446 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 msgr2=0x7f01dc0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 0x7f01dc0b8b00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f01f0109200 tx=0x7f01e0005f90 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 msgr2=0x7f01f019a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f01ec005940 tx=0x7f01ec004270 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 shutdown_connections 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01dc0b65f0 0x7f01dc0b8b00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f0109df0 0x7f01f019fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01f01094a0 0x7f01f019aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 --2- 172.21.15.16:0/2539991527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01f0103b60 0x7f01f019a480 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 >> 172.21.15.16:0/2539991527 conn(0x7f01f007b850 msgr2=0x7f01f0107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 shutdown_connections 2024-09-21T02:59:17.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:17.666+0000 7f01f8758640 1 -- 172.21.15.16:0/2539991527 wait complete. 2024-09-21T02:59:18.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:17 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2539991527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:59:18.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:17 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2539991527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:59:18.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:17 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2539991527' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 11} : dispatch 2024-09-21T02:59:18.381 INFO:tasks.cephadm.ceph_manager.ceph:need seq 296352743444 got 296352743446 for osd.11 2024-09-21T02:59:18.382 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 -- 172.21.15.16:0/1662116175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc944105330 msgr2=0x7fc94410d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 --2- 172.21.15.16:0/1662116175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc944105330 0x7fc94410d880 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fc938009900 tx=0x7fc93802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:18.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 -- 172.21.15.16:0/1662116175 shutdown_connections 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 --2- 172.21.15.16:0/1662116175 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc944105330 0x7fc94410d880 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 --2- 172.21.15.16:0/1662116175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc94410de40 0x7fc944110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 --2- 172.21.15.16:0/1662116175 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc9441049e0 0x7fc944104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 -- 172.21.15.16:0/1662116175 >> 172.21.15.16:0/1662116175 conn(0x7fc944079730 msgr2=0x7fc944079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.805+0000 7fc949eb1640 1 -- 172.21.15.16:0/1662116175 shutdown_connections 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.806+0000 7fc949eb1640 1 -- 172.21.15.16:0/1662116175 wait complete. 2024-09-21T02:59:18.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.806+0000 7fc949eb1640 1 Processor -- start 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.806+0000 7fc949eb1640 1 -- start start 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc944105330 0x7fc94419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 0x7fc94419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113aa0 con 0x7fc9441049e0 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113920 con 0x7fc94410de40 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc949eb1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc944113c20 con 0x7fc944105330 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:18.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40804/0 (socket says 172.21.15.16:40804) 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 -- 172.21.15.16:0/2916232812 learned_addr learned my addr 172.21.15.16:0/2916232812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc942ffd640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc944105330 0x7fc94419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc943fff640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 0x7fc94419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 -- 172.21.15.16:0/2916232812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc944105330 msgr2=0x7fc94419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc944105330 0x7fc94419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 -- 172.21.15.16:0/2916232812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 msgr2=0x7fc94419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 0x7fc94419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 -- 172.21.15.16:0/2916232812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9441a04c0 con 0x7fc9441049e0 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc943fff640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 0x7fc94419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.807+0000 7fc9437fe640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc93400d950 tx=0x7fc93400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:18.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.808+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc934017070 con 0x7fc9441049e0 2024-09-21T02:59:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.808+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9441a07b0 con 0x7fc9441049e0 2024-09-21T02:59:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.808+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc934009c30 con 0x7fc9441049e0 2024-09-21T02:59:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.808+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc93400f620 con 0x7fc9441049e0 2024-09-21T02:59:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.808+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9441a6f10 con 0x7fc9441049e0 2024-09-21T02:59:18.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.809+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc94406c240 con 0x7fc9441049e0 2024-09-21T02:59:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.810+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fc934010460 con 0x7fc9441049e0 2024-09-21T02:59:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.811+0000 7fc940ff9640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 0x7fc92c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.811+0000 7fc942ffd640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 0x7fc92c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.811+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fc9340e7250 con 0x7fc9441049e0 2024-09-21T02:59:18.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.812+0000 7fc942ffd640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 0x7fc92c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc938009230 tx=0x7fc938031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:18.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:18.816+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc9340ad8a0 con 0x7fc9441049e0 2024-09-21T02:59:19.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.008+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc944108f40 con 0x7fc9441049e0 2024-09-21T02:59:19.009 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:18 smithi016 ceph-mon[31253]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:19.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.008+0000 7fc940ff9640 1 -- 172.21.15.16:0/2916232812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc934009de0 con 0x7fc9441049e0 2024-09-21T02:59:19.010 INFO:teuthology.orchestra.run.smithi016.stdout:60129542216 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 msgr2=0x7fc92c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 0x7fc92c0b8b00 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fc938009230 tx=0x7fc938031400 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 msgr2=0x7fc94419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fc93400d950 tx=0x7fc93400de10 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 shutdown_connections 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc92c0b65f0 0x7fc92c0b8b00 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc94410de40 0x7fc94419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc944105330 0x7fc94419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 --2- 172.21.15.16:0/2916232812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc9441049e0 0x7fc94419a550 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:19.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 >> 172.21.15.16:0/2916232812 conn(0x7fc944079730 msgr2=0x7fc9440ff750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 shutdown_connections 2024-09-21T02:59:19.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:19.010+0000 7fc949eb1640 1 -- 172.21.15.16:0/2916232812 wait complete. 2024-09-21T02:59:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:18 smithi018 ceph-mon[34587]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:18 smithi123 ceph-mon[34645]: pgmap v287: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:19.251 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:19.253 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:19.254 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:19.256 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:19.298 INFO:tasks.cephadm.ceph_manager.ceph:need seq 231928234020 got 231928234022 for osd.8 2024-09-21T02:59:19.298 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:19.434 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:20.120 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:19 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2916232812' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 -- 172.21.15.16:0/1078024870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108cf0 msgr2=0x7f43b810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 --2- 172.21.15.16:0/1078024870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108cf0 0x7f43b810f520 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f43b0009ca0 tx=0x7f43b002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 -- 172.21.15.16:0/1078024870 shutdown_connections 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 --2- 172.21.15.16:0/1078024870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8108cf0 0x7f43b810f520 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 --2- 172.21.15.16:0/1078024870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b81083a0 0x7f43b81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 --2- 172.21.15.16:0/1078024870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b8102a60 0x7f43b8102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 -- 172.21.15.16:0/1078024870 >> 172.21.15.16:0/1078024870 conn(0x7f43b8079730 msgr2=0x7f43b8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 -- 172.21.15.16:0/1078024870 shutdown_connections 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.118+0000 7f43beec6640 1 -- 172.21.15.16:0/1078024870 wait complete. 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 Processor -- start 2024-09-21T02:59:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 -- start start 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81083a0 0x7f43b81981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 0x7f43b8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8112960 con 0x7f43b8102a60 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81127e0 con 0x7f43b8108cf0 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43beec6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8112ae0 con 0x7f43b81083a0 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43bcc62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43bcc62640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40830/0 (socket says 172.21.15.16:40830) 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.119+0000 7f43bcc62640 1 -- 172.21.15.16:0/1496746446 learned_addr learned my addr 172.21.15.16:0/1496746446 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43b7fff640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81083a0 0x7f43b81981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bd463640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 0x7f43b8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 -- 172.21.15.16:0/1496746446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81083a0 msgr2=0x7f43b81981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81083a0 0x7f43b81981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 -- 172.21.15.16:0/1496746446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 msgr2=0x7f43b8198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 0x7f43b8198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 -- 172.21.15.16:0/1496746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b8198e10 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bd463640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 0x7f43b8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43bcc62640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c920 tx=0x7f43a800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8007c70 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81b5650 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8007e10 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8018820 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.120+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81b5b00 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.121+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43b8102e70 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.124+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f43a800f450 con 0x7f43b8102a60 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.125+0000 7f43b5ffb640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 0x7f43a00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.125+0000 7f43b7fff640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 0x7f43a00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.125+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f43a80e79d0 con 0x7f43b8102a60 2024-09-21T02:59:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.125+0000 7f43b7fff640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 0x7f43a00b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43b806a340 tx=0x7f43ac005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:20.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.128+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f43a8014030 con 0x7f43b8102a60 2024-09-21T02:59:20.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:19 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2916232812' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:59:20.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:19 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2916232812' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 1} : dispatch 2024-09-21T02:59:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.318+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f43b8199350 con 0x7f43b8102a60 2024-09-21T02:59:20.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.318+0000 7f43b5ffb640 1 -- 172.21.15.16:0/1496746446 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f43a80ae050 con 0x7f43b8102a60 2024-09-21T02:59:20.319 INFO:teuthology.orchestra.run.smithi016.stdout:274877906972 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 msgr2=0x7f43a00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 0x7f43a00b8b00 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f43b806a340 tx=0x7f43ac005f90 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 msgr2=0x7f43b819fdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f43a800c920 tx=0x7f43a800cde0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 shutdown_connections 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43a00b65f0 0x7f43a00b8b00 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43b8108cf0 0x7f43b8198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f43b81083a0 0x7f43b81981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 --2- 172.21.15.16:0/1496746446 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43b8102a60 0x7f43b819fdb0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.320+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 >> 172.21.15.16:0/1496746446 conn(0x7f43b8079730 msgr2=0x7f43b806b000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.321+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 shutdown_connections 2024-09-21T02:59:20.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.321+0000 7f43beec6640 1 -- 172.21.15.16:0/1496746446 wait complete. 2024-09-21T02:59:20.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1082092115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 msgr2=0x7fe6a4104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1082092115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a4104dc0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fe694009900 tx=0x7fe69402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1082092115 shutdown_connections 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1082092115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a410de10 0x7fe6a4110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1082092115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a4105300 0x7fe6a410d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1082092115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a4104dc0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1082092115 >> 172.21.15.16:0/1082092115 conn(0x7fe6a4079730 msgr2=0x7fe6a4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1082092115 shutdown_connections 2024-09-21T02:59:20.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.380+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1082092115 wait complete. 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 Processor -- start 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 -- start start 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a4105300 0x7fe6a4102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a410de10 0x7fe6a4103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4113a70 con 0x7fe6a41049b0 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a41138f0 con 0x7fe6a410de10 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6aa0e9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe6a4113bf0 con 0x7fe6a4105300 2024-09-21T02:59:20.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6a37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40852/0 (socket says 172.21.15.16:40852) 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.381+0000 7fe6a37fe640 1 -- 172.21.15.16:0/1954301813 learned_addr learned my addr 172.21.15.16:0/1954301813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a3fff640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a410de10 0x7fe6a4103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a2ffd640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a4105300 0x7fe6a4102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 -- 172.21.15.16:0/1954301813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a4105300 msgr2=0x7fe6a4102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a4105300 0x7fe6a4102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 -- 172.21.15.16:0/1954301813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a410de10 msgr2=0x7fe6a4103170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a410de10 0x7fe6a4103170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 -- 172.21.15.16:0/1954301813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6a41a86f0 con 0x7fe6a41049b0 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a37fe640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe694031a90 tx=0x7fe694004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:20.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe69403f030 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe694003cb0 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6a41a8980 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe694035070 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.382+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6a41a8e60 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.384+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6a4104dc0 con 0x7fe6a41049b0 2024-09-21T02:59:20.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.385+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe694042030 con 0x7fe6a41049b0 2024-09-21T02:59:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.386+0000 7fe6a0ff9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 0x7fe68c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.386+0000 7fe6a2ffd640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 0x7fe68c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:20.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.386+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fe694107b30 con 0x7fe6a41049b0 2024-09-21T02:59:20.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.387+0000 7fe6a2ffd640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 0x7fe68c0b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe6a410fac0 tx=0x7fe69800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:20.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.390+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe6940d2560 con 0x7fe6a41049b0 2024-09-21T02:59:20.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.579+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7fe6a41a91f0 con 0x7fe6a41049b0 2024-09-21T02:59:20.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.579+0000 7fe6a0ff9640 1 -- 172.21.15.16:0/1954301813 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fe6940d74a0 con 0x7fe6a41049b0 2024-09-21T02:59:20.581 INFO:teuthology.orchestra.run.smithi016.stdout:111669149759 2024-09-21T02:59:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.581+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 msgr2=0x7fe68c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.581+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 0x7fe68c0b8b00 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fe6a410fac0 tx=0x7fe69800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 msgr2=0x7fe6a41026f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7fe694031a90 tx=0x7fe694004270 comp rx=0 tx=0).stop 2024-09-21T02:59:20.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 shutdown_connections 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe68c0b65f0 0x7fe68c0b8b00 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe6a410de10 0x7fe6a4103170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a4105300 0x7fe6a4102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 --2- 172.21.15.16:0/1954301813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe6a41049b0 0x7fe6a41026f0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 >> 172.21.15.16:0/1954301813 conn(0x7fe6a4079730 msgr2=0x7fe6a4106820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 shutdown_connections 2024-09-21T02:59:20.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:20.582+0000 7fe6aa0e9640 1 -- 172.21.15.16:0/1954301813 wait complete. 2024-09-21T02:59:21.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:20 smithi016 ceph-mon[31253]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:21.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1496746446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:59:21.168 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:20 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1954301813' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:59:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:20 smithi018 ceph-mon[34587]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1496746446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:59:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:20 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1954301813' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:59:21.197 INFO:tasks.cephadm.ceph_manager.ceph:need seq 274877906971 got 274877906972 for osd.10 2024-09-21T02:59:21.198 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:20 smithi123 ceph-mon[34645]: pgmap v288: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1496746446' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 10} : dispatch 2024-09-21T02:59:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:20 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1954301813' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 3} : dispatch 2024-09-21T02:59:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.832+0000 7f906d61d640 1 -- 172.21.15.16:0/4108858187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906810de10 msgr2=0x7f9068110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.832+0000 7f906d61d640 1 --2- 172.21.15.16:0/4108858187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906810de10 0x7f9068110300 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f9060009ca0 tx=0x7f906002a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.832+0000 7f906d61d640 1 -- 172.21.15.16:0/4108858187 shutdown_connections 2024-09-21T02:59:21.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 --2- 172.21.15.16:0/4108858187 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f906810de10 0x7f9068110300 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 --2- 172.21.15.16:0/4108858187 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 0x7f906810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 --2- 172.21.15.16:0/4108858187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90681049b0 0x7f9068104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 -- 172.21.15.16:0/4108858187 >> 172.21.15.16:0/4108858187 conn(0x7f9068079730 msgr2=0x7f9068079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 -- 172.21.15.16:0/4108858187 shutdown_connections 2024-09-21T02:59:21.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 -- 172.21.15.16:0/4108858187 wait complete. 2024-09-21T02:59:21.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 Processor -- start 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.833+0000 7f906d61d640 1 -- start start 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 0x7f906819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f906810de10 0x7f906819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9068113a70 con 0x7f90681049b0 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f90681138f0 con 0x7f9068105300 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f906d61d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9068113bf0 con 0x7f906810de10 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40876/0 (socket says 172.21.15.16:40876) 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 -- 172.21.15.16:0/3716487696 learned_addr learned my addr 172.21.15.16:0/3716487696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f90677fe640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f906810de10 0x7f906819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f905e7fc640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 0x7f906819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:21.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 -- 172.21.15.16:0/3716487696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f906810de10 msgr2=0x7f906819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f906810de10 0x7f906819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 -- 172.21.15.16:0/3716487696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 msgr2=0x7f906819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 0x7f906819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.834+0000 7f9066ffd640 1 -- 172.21.15.16:0/3716487696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f90681a0410 con 0x7f90681049b0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f9066ffd640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f905800d770 tx=0x7f905800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058017070 con 0x7f90681049b0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9058009c30 con 0x7f90681049b0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f90681a0700 con 0x7f90681049b0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9058003cc0 con 0x7f90681049b0 2024-09-21T02:59:21.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.835+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90681a6e60 con 0x7f90681049b0 2024-09-21T02:59:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.836+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9068104dc0 con 0x7f90681049b0 2024-09-21T02:59:21.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.837+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f9058004020 con 0x7f90681049b0 2024-09-21T02:59:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.839+0000 7f9064ff9640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 0x7f90540b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.839+0000 7f905e7fc640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 0x7f90540b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:21.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.839+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f90580e6ff0 con 0x7f90681049b0 2024-09-21T02:59:21.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.840+0000 7f905e7fc640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 0x7f90540b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9050005fd0 tx=0x7f9050005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:21.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:21.843+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9058009de0 con 0x7f90681049b0 2024-09-21T02:59:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.035+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f90681a7150 con 0x7f90681049b0 2024-09-21T02:59:22.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.035+0000 7f9064ff9640 1 -- 172.21.15.16:0/3716487696 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f90580ad5e0 con 0x7f90681049b0 2024-09-21T02:59:22.037 INFO:teuthology.orchestra.run.smithi016.stdout:253403070498 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 msgr2=0x7f90540b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 0x7f90540b8b00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9050005fd0 tx=0x7f9050005850 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 msgr2=0x7f906819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f905800d770 tx=0x7f905800dc30 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 shutdown_connections 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f90540b65f0 0x7f90540b8b00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f906810de10 0x7f906819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9068105300 0x7f906819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 --2- 172.21.15.16:0/3716487696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f90681049b0 0x7f906819a4a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 >> 172.21.15.16:0/3716487696 conn(0x7f9068079730 msgr2=0x7f90680ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 shutdown_connections 2024-09-21T02:59:22.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.037+0000 7f906d61d640 1 -- 172.21.15.16:0/3716487696 wait complete. 2024-09-21T02:59:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 -- 172.21.15.16:0/990459412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3770102ae0 msgr2=0x7f3770102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 --2- 172.21.15.16:0/990459412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3770102ae0 0x7f3770102ef0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f3758009900 tx=0x7f375802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 -- 172.21.15.16:0/990459412 shutdown_connections 2024-09-21T02:59:22.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 --2- 172.21.15.16:0/990459412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770108d20 0x7f377010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 --2- 172.21.15.16:0/990459412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37701083d0 0x7f37701087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 --2- 172.21.15.16:0/990459412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3770102ae0 0x7f3770102ef0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 -- 172.21.15.16:0/990459412 >> 172.21.15.16:0/990459412 conn(0x7f3770079760 msgr2=0x7f3770079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 -- 172.21.15.16:0/990459412 shutdown_connections 2024-09-21T02:59:22.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.182+0000 7f3776fb1640 1 -- 172.21.15.16:0/990459412 wait complete. 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 Processor -- start 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 -- start start 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 0x7f377010ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3770108d20 0x7f3770110a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3770113ab0 con 0x7f37701083d0 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3770113930 con 0x7f3770102ae0 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.183+0000 7f3776fb1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3770113c30 con 0x7f3770108d20 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40892/0 (socket says 172.21.15.16:40892) 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 -- 172.21.15.16:0/2694965207 learned_addr learned my addr 172.21.15.16:0/2694965207 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:22.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f3774d4d640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 0x7f377010ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f377554e640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3770108d20 0x7f3770110a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 -- 172.21.15.16:0/2694965207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3770108d20 msgr2=0x7f3770110a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3770108d20 0x7f3770110a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 -- 172.21.15.16:0/2694965207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 msgr2=0x7f377010ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 0x7f377010ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 -- 172.21.15.16:0/2694965207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37701a8800 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f3774d4d640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 0x7f377010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376ffff640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f376000ea50 tx=0x7f376000ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376000cdc0 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37701a8af0 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.184+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3760005020 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.185+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f37600116b0 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.185+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37701a9030 con 0x7f37701083d0 2024-09-21T02:59:22.189 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.186+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3770103d10 con 0x7f37701083d0 2024-09-21T02:59:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.187+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f37600051d0 con 0x7f37701083d0 2024-09-21T02:59:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.188+0000 7f376dffb640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 0x7f375c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.188+0000 7f3774d4d640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 0x7f375c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:22.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.188+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f37600e77b0 con 0x7f37701083d0 2024-09-21T02:59:22.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.188+0000 7f3774d4d640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 0x7f375c0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:22.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.192+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3760016030 con 0x7f37701083d0 2024-09-21T02:59:22.218 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:22.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.392+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f37701112e0 con 0x7f37701083d0 2024-09-21T02:59:22.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.392+0000 7f376dffb640 1 -- 172.21.15.16:0/2694965207 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f37600b21e0 con 0x7f37701083d0 2024-09-21T02:59:22.394 INFO:teuthology.orchestra.run.smithi016.stdout:38654705743 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.394+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 msgr2=0x7f375c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.394+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 0x7f375c0b8b00 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3758000c00 tx=0x7f3758031400 comp rx=0 tx=0).stop 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.394+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 msgr2=0x7f3770110520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.394+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f376000ea50 tx=0x7f376000ef10 comp rx=0 tx=0).stop 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 shutdown_connections 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f375c0b65f0 0x7f375c0b8b00 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f37701083d0 0x7f3770110520 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3770108d20 0x7f3770110a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 --2- 172.21.15.16:0/2694965207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3770102ae0 0x7f377010ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 >> 172.21.15.16:0/2694965207 conn(0x7f3770079760 msgr2=0x7f3770101510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 shutdown_connections 2024-09-21T02:59:22.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:22.395+0000 7f3776fb1640 1 -- 172.21.15.16:0/2694965207 wait complete. 2024-09-21T02:59:22.641 INFO:tasks.cephadm.ceph_manager.ceph:need seq 60129542215 got 60129542216 for osd.1 2024-09-21T02:59:22.641 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:22 smithi018 ceph-mon[34587]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3716487696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:59:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2694965207' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:59:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:22 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:22 smithi123 ceph-mon[34645]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3716487696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:59:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2694965207' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:59:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:22 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:23.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:22 smithi016 ceph-mon[31253]: pgmap v289: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:23.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3716487696' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 9} : dispatch 2024-09-21T02:59:23.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2694965207' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 0} : dispatch 2024-09-21T02:59:23.262 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:22 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T02:59:23.290 INFO:tasks.cephadm.ceph_manager.ceph:need seq 38654705740 got 38654705743 for osd.0 2024-09-21T02:59:23.290 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:24.091 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070495 got 253403070498 for osd.9 2024-09-21T02:59:24.145 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:24.723 INFO:tasks.cephadm.ceph_manager.ceph:need seq 111669149756 got 111669149759 for osd.3 2024-09-21T02:59:24.723 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:24 smithi016 ceph-mon[31253]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T02:59:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:24 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:24 smithi018 ceph-mon[34587]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T02:59:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:24 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:24 smithi123 ceph-mon[34645]: pgmap v290: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 1 op/s 2024-09-21T02:59:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:24 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- 172.21.15.16:0/904515147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f0109df0 msgr2=0x7f48f0110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 --2- 172.21.15.16:0/904515147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f0109df0 0x7f48f0110620 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f48e8009ca0 tx=0x7f48e802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- 172.21.15.16:0/904515147 shutdown_connections 2024-09-21T02:59:25.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 --2- 172.21.15.16:0/904515147 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f0109df0 0x7f48f0110620 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 --2- 172.21.15.16:0/904515147 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f01094a0 0x7f48f01098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 --2- 172.21.15.16:0/904515147 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0103b60 0x7f48f0103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- 172.21.15.16:0/904515147 >> 172.21.15.16:0/904515147 conn(0x7f48f007b850 msgr2=0x7f48f007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- 172.21.15.16:0/904515147 shutdown_connections 2024-09-21T02:59:25.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- 172.21.15.16:0/904515147 wait complete. 2024-09-21T02:59:25.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 Processor -- start 2024-09-21T02:59:25.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.260+0000 7f48f653e640 1 -- start start 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 0x7f48f0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0109df0 0x7f48f0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0113ab0 con 0x7f48f01094a0 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0113930 con 0x7f48f0109df0 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f653e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48f0113c30 con 0x7f48f0103b60 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57670/0 (socket says 172.21.15.16:57670) 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 -- 172.21.15.16:0/1839782040 learned_addr learned my addr 172.21.15.16:0/1839782040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48f4adb640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0109df0 0x7f48f0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48effff640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 0x7f48f0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 -- 172.21.15.16:0/1839782040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 msgr2=0x7f48f0075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 0x7f48f0075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 -- 172.21.15.16:0/1839782040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0109df0 msgr2=0x7f48f0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0109df0 0x7f48f0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48ef7fe640 1 -- 172.21.15.16:0/1839782040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48f01a8740 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.261+0000 7f48effff640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 0x7f48f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48ef7fe640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d740 tx=0x7f48e400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e4017070 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48e400de50 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48f01a8a30 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f48e400f620 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.262+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48f01a8f70 con 0x7f48f01094a0 2024-09-21T02:59:25.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.263+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48f0103f70 con 0x7f48f01094a0 2024-09-21T02:59:25.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.265+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f48e401a070 con 0x7f48f01094a0 2024-09-21T02:59:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.265+0000 7f48ed7fa640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 0x7f48d80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.266+0000 7f48effff640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 0x7f48d80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.266+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f48e40e6f80 con 0x7f48f01094a0 2024-09-21T02:59:25.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.266+0000 7f48effff640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 0x7f48d80b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005fd0 tx=0x7f48e0005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:25.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.269+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f48e40ad5d0 con 0x7f48f01094a0 2024-09-21T02:59:25.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.462+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f48f002e640 con 0x7f48f01094a0 2024-09-21T02:59:25.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.462+0000 7f48ed7fa640 1 -- 172.21.15.16:0/1839782040 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f48e40b19b0 con 0x7f48f01094a0 2024-09-21T02:59:25.465 INFO:teuthology.orchestra.run.smithi016.stdout:184683593777 2024-09-21T02:59:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 msgr2=0x7f48d80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 0x7f48d80b8b00 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f48e0005fd0 tx=0x7f48e0005850 comp rx=0 tx=0).stop 2024-09-21T02:59:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 msgr2=0x7f48f0075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f48e400d740 tx=0x7f48e400dc00 comp rx=0 tx=0).stop 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 shutdown_connections 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48d80b65f0 0x7f48d80b8b00 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f48f01094a0 0x7f48f0075c50 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48f0109df0 0x7f48f0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 --2- 172.21.15.16:0/1839782040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48f0103b60 0x7f48f0075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 >> 172.21.15.16:0/1839782040 conn(0x7f48f007b850 msgr2=0x7f48f01054b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 shutdown_connections 2024-09-21T02:59:25.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.465+0000 7f48f653e640 1 -- 172.21.15.16:0/1839782040 wait complete. 2024-09-21T02:59:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1810928797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18076100 msgr2=0x7ffa1810fbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1810928797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18076100 0x7ffa1810fbe0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7ffa08009900 tx=0x7ffa0802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1810928797 shutdown_connections 2024-09-21T02:59:25.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1810928797 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18076100 0x7ffa1810fbe0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1810928797 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa18110120 0x7ffa18112540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1810928797 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa180757b0 0x7ffa18075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1810928797 >> 172.21.15.16:0/1810928797 conn(0x7ffa1807b910 msgr2=0x7ffa1807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1810928797 shutdown_connections 2024-09-21T02:59:25.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1810928797 wait complete. 2024-09-21T02:59:25.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.876+0000 7ffa1fe0b640 1 Processor -- start 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 -- start start 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa180757b0 0x7ffa1819c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa18076100 0x7ffa1819cc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18115c90 con 0x7ffa18110120 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18115b10 con 0x7ffa180757b0 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1fe0b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ffa18115e10 con 0x7ffa18076100 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57694/0 (socket says 172.21.15.16:57694) 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 -- 172.21.15.16:0/1327656597 learned_addr learned my addr 172.21.15.16:0/1327656597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1dba7640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa180757b0 0x7ffa1819c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1d3a6640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa18076100 0x7ffa1819cc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 -- 172.21.15.16:0/1327656597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa18076100 msgr2=0x7ffa1819cc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa18076100 0x7ffa1819cc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 -- 172.21.15.16:0/1327656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa180757b0 msgr2=0x7ffa1819c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa180757b0 0x7ffa1819c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.877+0000 7ffa1e3a8640 1 -- 172.21.15.16:0/1327656597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa181a2630 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa1e3a8640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009fd0 tx=0x7ffa1000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10017070 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa181a2920 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa10003e20 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa10010460 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.878+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa181a9080 con 0x7ffa18110120 2024-09-21T02:59:25.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.879+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa18075bc0 con 0x7ffa18110120 2024-09-21T02:59:25.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.881+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ffa10003970 con 0x7ffa18110120 2024-09-21T02:59:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.882+0000 7ffa0effd640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 0x7ffa040b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.882+0000 7ffa1dba7640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 0x7ffa040b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.882+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7ffa100e73f0 con 0x7ffa18110120 2024-09-21T02:59:25.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.882+0000 7ffa1dba7640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 0x7ffa040b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009bd0 tx=0x7ffa14005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:25.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:25.885+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ffa100ad9e0 con 0x7ffa18110120 2024-09-21T02:59:26.037 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:25 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1839782040' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:59:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.075+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7ffa181a9250 con 0x7ffa18110120 2024-09-21T02:59:26.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.076+0000 7ffa0effd640 1 -- 172.21.15.16:0/1327656597 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7ffa100b1dc0 con 0x7ffa18110120 2024-09-21T02:59:26.078 INFO:teuthology.orchestra.run.smithi016.stdout:137438953530 2024-09-21T02:59:26.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 msgr2=0x7ffa040b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 0x7ffa040b8b00 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffa14009bd0 tx=0x7ffa14005f90 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 msgr2=0x7ffa181a1f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7ffa10009fd0 tx=0x7ffa1000b260 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 shutdown_connections 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ffa040b65f0 0x7ffa040b8b00 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ffa18110120 0x7ffa181a1f00 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffa18076100 0x7ffa1819cc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 --2- 172.21.15.16:0/1327656597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ffa180757b0 0x7ffa1819c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 >> 172.21.15.16:0/1327656597 conn(0x7ffa1807b910 msgr2=0x7ffa18109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 shutdown_connections 2024-09-21T02:59:26.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.078+0000 7ffa1fe0b640 1 -- 172.21.15.16:0/1327656597 wait complete. 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 -- 172.21.15.16:0/1064361836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430103b90 msgr2=0x7f1430103fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 --2- 172.21.15.16:0/1064361836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430103b90 0x7f1430103fa0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f1420009900 tx=0x7f142002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 -- 172.21.15.16:0/1064361836 shutdown_connections 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 --2- 172.21.15.16:0/1064361836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430109e20 0x7f1430110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 --2- 172.21.15.16:0/1064361836 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f14301094d0 0x7f14301098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 --2- 172.21.15.16:0/1064361836 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1430103b90 0x7f1430103fa0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 -- 172.21.15.16:0/1064361836 >> 172.21.15.16:0/1064361836 conn(0x7f14300ff600 msgr2=0x7f1430101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 -- 172.21.15.16:0/1064361836 shutdown_connections 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.126+0000 7f14360bb640 1 -- 172.21.15.16:0/1064361836 wait complete. 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 Processor -- start 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 -- start start 2024-09-21T02:59:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430103b90 0x7f143019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1430109e20 0x7f143019fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1430113ae0 con 0x7f14301094d0 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1430113960 con 0x7f1430109e20 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f14360bb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1430113c60 con 0x7f1430103b90 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.127+0000 7f142effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57710/0 (socket says 172.21.15.16:57710) 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 -- 172.21.15.16:0/1575479605 learned_addr learned my addr 172.21.15.16:0/1575479605 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142ffff640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1430109e20 0x7f143019fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142f7fe640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430103b90 0x7f143019a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 -- 172.21.15.16:0/1575479605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430103b90 msgr2=0x7f143019a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430103b90 0x7f143019a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 -- 172.21.15.16:0/1575479605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1430109e20 msgr2=0x7f143019fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1430109e20 0x7f143019fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 -- 172.21.15.16:0/1575479605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14301a03e0 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142effd640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f142400d740 tx=0x7f142400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424017070 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14301a06d0 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142400de50 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142400f620 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.128+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a6e30 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.129+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1430103fa0 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.131+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f142401a070 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.132+0000 7f142cff9640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 0x7f14180b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.132+0000 7f142f7fe640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 0x7f14180b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.132+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f14240e6eb0 con 0x7f14301094d0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.132+0000 7f142f7fe640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 0x7f14180b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1420031440 tx=0x7f1420009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.135+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f14240b18e0 con 0x7f14301094d0 2024-09-21T02:59:26.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:25 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1839782040' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:59:26.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:25 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1839782040' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 6} : dispatch 2024-09-21T02:59:26.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.325+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f143010b660 con 0x7f14301094d0 2024-09-21T02:59:26.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.326+0000 7f142cff9640 1 -- 172.21.15.16:0/1575479605 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f14240b6820 con 0x7f14301094d0 2024-09-21T02:59:26.328 INFO:teuthology.orchestra.run.smithi016.stdout:163208757302 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 msgr2=0x7f14180b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 0x7f14180b8b00 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1420031440 tx=0x7f1420009560 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 msgr2=0x7f143019aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f142400d740 tx=0x7f142400dc00 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 shutdown_connections 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f14180b65f0 0x7f14180b8b00 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f14301094d0 0x7f143019aa10 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1430109e20 0x7f143019fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 --2- 172.21.15.16:0/1575479605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1430103b90 0x7f143019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 >> 172.21.15.16:0/1575479605 conn(0x7f14300ff600 msgr2=0x7f1430101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:26.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 shutdown_connections 2024-09-21T02:59:26.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:26.328+0000 7f14360bb640 1 -- 172.21.15.16:0/1575479605 wait complete. 2024-09-21T02:59:26.708 INFO:tasks.cephadm.ceph_manager.ceph:need seq 137438953527 got 137438953530 for osd.4 2024-09-21T02:59:26.708 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:27.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:26 smithi016 ceph-mon[31253]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:27.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1327656597' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:59:27.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:26 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1575479605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:59:27.147 INFO:tasks.cephadm.ceph_manager.ceph:need seq 184683593773 got 184683593777 for osd.6 2024-09-21T02:59:27.147 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:26 smithi018 ceph-mon[34587]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:26 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1327656597' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:59:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:26 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1575479605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:59:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:26 smithi123 ceph-mon[34645]: pgmap v291: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:26 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1327656597' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 4} : dispatch 2024-09-21T02:59:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:26 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1575479605' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 5} : dispatch 2024-09-21T02:59:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 -- 172.21.15.16:0/3419582364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c50105190 msgr2=0x7f0c501055a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 --2- 172.21.15.16:0/3419582364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c50105190 0x7f0c501055a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009900 tx=0x7f0c3c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 -- 172.21.15.16:0/3419582364 shutdown_connections 2024-09-21T02:59:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 --2- 172.21.15.16:0/3419582364 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c50105190 0x7f0c501055a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 --2- 172.21.15.16:0/3419582364 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105ae0 0x7f0c5010f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 --2- 172.21.15.16:0/3419582364 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c501031c0 0x7f0c501035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 -- 172.21.15.16:0/3419582364 >> 172.21.15.16:0/3419582364 conn(0x7f0c50077c60 msgr2=0x7f0c500780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 -- 172.21.15.16:0/3419582364 shutdown_connections 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.320+0000 7f0c56df2640 1 -- 172.21.15.16:0/3419582364 wait complete. 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.321+0000 7f0c56df2640 1 Processor -- start 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.321+0000 7f0c56df2640 1 -- start start 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.321+0000 7f0c56df2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.321+0000 7f0c56df2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105190 0x7f0c50199970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57734/0 (socket says 172.21.15.16:57734) 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c56df2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 0x7f0c5019ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 -- 172.21.15.16:0/87157593 learned_addr learned my addr 172.21.15.16:0/87157593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50112980 con 0x7f0c501031c0 2024-09-21T02:59:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50112800 con 0x7f0c50105ae0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0c50112b00 con 0x7f0c50105190 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c4ffff640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105190 0x7f0c50199970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 -- 172.21.15.16:0/87157593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105190 msgr2=0x7f0c50199970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105190 0x7f0c50199970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c5538f640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 0x7f0c5019ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 -- 172.21.15.16:0/87157593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 msgr2=0x7f0c5019ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 0x7f0c5019ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c5019f340 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c5538f640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 0x7f0c5019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c54b8e640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400d950 tx=0x7f0c4400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c44017070 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.322+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c5019f630 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.323+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c44009c30 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.323+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c4400f620 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.323+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c501a5d90 con 0x7f0c501031c0 2024-09-21T02:59:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.324+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c501035d0 con 0x7f0c501031c0 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.325+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f0c44005030 con 0x7f0c501031c0 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.326+0000 7f0c4dffb640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 0x7f0c400b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.326+0000 7f0c4ffff640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 0x7f0c400b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.326+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f0c440e7190 con 0x7f0c501031c0 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.327+0000 7f0c4ffff640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 0x7f0c400b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.329+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0c440ad860 con 0x7f0c501031c0 2024-09-21T02:59:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.522+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f0c5002e640 con 0x7f0c501031c0 2024-09-21T02:59:27.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.523+0000 7f0c4dffb640 1 -- 172.21.15.16:0/87157593 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0c44009de0 con 0x7f0c501031c0 2024-09-21T02:59:27.526 INFO:teuthology.orchestra.run.smithi016.stdout:81604378695 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 msgr2=0x7f0c400b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 0x7f0c400b8b80 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c000c00 tx=0x7f0c3c031400 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 msgr2=0x7f0c501993d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f0c4400d950 tx=0x7f0c4400de10 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 shutdown_connections 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0c400b6670 0x7f0c400b8b80 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0c50105ae0 0x7f0c5019ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0c50105190 0x7f0c50199970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 --2- 172.21.15.16:0/87157593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0c501031c0 0x7f0c501993d0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.525+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 >> 172.21.15.16:0/87157593 conn(0x7f0c50077c60 msgr2=0x7f0c501064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.526+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 shutdown_connections 2024-09-21T02:59:27.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:27.526+0000 7f0c56df2640 1 -- 172.21.15.16:0/87157593 wait complete. 2024-09-21T02:59:28.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:27 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/87157593' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:59:28.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:27 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/87157593' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:59:28.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:27 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/87157593' entity='client.admin' cmd={"prefix": "osd last-stat-seq", "id": 2} : dispatch 2024-09-21T02:59:28.235 INFO:tasks.cephadm.ceph_manager.ceph:need seq 81604378692 got 81604378695 for osd.2 2024-09-21T02:59:28.235 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:28.654 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757299 got 163208757302 for osd.5 2024-09-21T02:59:28.654 DEBUG:teuthology.parallel:result is None 2024-09-21T02:59:28.654 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-09-21T02:59:28.654 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:59:29.148 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:28 smithi016 ceph-mon[31253]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:28 smithi018 ceph-mon[34587]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:28 smithi123 ceph-mon[34645]: pgmap v292: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:29.561 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 -- 172.21.15.16:0/3345579872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1049f0 msgr2=0x7ff72c104e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 --2- 172.21.15.16:0/3345579872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1049f0 0x7ff72c104e00 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7ff728009900 tx=0x7ff72802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 -- 172.21.15.16:0/3345579872 shutdown_connections 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 --2- 172.21.15.16:0/3345579872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c10de50 0x7ff72c110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 --2- 172.21.15.16:0/3345579872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c105340 0x7ff72c10d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 --2- 172.21.15.16:0/3345579872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c1049f0 0x7ff72c104e00 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 -- 172.21.15.16:0/3345579872 >> 172.21.15.16:0/3345579872 conn(0x7ff72c079750 msgr2=0x7ff72c079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 -- 172.21.15.16:0/3345579872 shutdown_connections 2024-09-21T02:59:30.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 -- 172.21.15.16:0/3345579872 wait complete. 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.058+0000 7ff733df5640 1 Processor -- start 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 -- start start 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 0x7ff72c19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c105340 0x7ff72c19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113ab0 con 0x7ff72c10de50 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113930 con 0x7ff72c1049f0 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff733df5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff72c113c30 con 0x7ff72c105340 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57756/0 (socket says 172.21.15.16:57756) 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 -- 172.21.15.16:0/3437192287 learned_addr learned my addr 172.21.15.16:0/3437192287 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff731b91640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 0x7ff72c19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 -- 172.21.15.16:0/3437192287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c105340 msgr2=0x7ff72c19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c105340 0x7ff72c19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 -- 172.21.15.16:0/3437192287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 msgr2=0x7ff72c19a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 0x7ff72c19a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 -- 172.21.15.16:0/3437192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff72c1a0450 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff731b91640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 0x7ff72c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.059+0000 7ff732392640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fd0 tx=0x7ff72400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.060+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724017070 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.060+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff724003e20 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.060+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff72c1a0740 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.060+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724010460 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.060+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff72c1a6ea0 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.061+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff72c104e00 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.062+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7ff724003970 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.063+0000 7ff722ffd640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 0x7ff7180b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.063+0000 7ff731b91640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 0x7ff7180b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.064+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7ff7240e73f0 con 0x7ff72c10de50 2024-09-21T02:59:30.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.064+0000 7ff731b91640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 0x7ff7180b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7280096c0 tx=0x7ff728031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:30.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.069+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff7240ad9e0 con 0x7ff72c10de50 2024-09-21T02:59:30.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.277+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7ff72c108080 con 0x7ff7180b6670 2024-09-21T02:59:30.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.281+0000 7ff722ffd640 1 -- 172.21.15.16:0/3437192287 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258617 (secure 0 0 0) 0x7ff72c108080 con 0x7ff7180b6670 2024-09-21T02:59:30.286 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:30.287 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 msgr2=0x7ff7180b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 0x7ff7180b8b80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ff7280096c0 tx=0x7ff728031400 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 msgr2=0x7ff72c19fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ff724009fd0 tx=0x7ff72400b260 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 shutdown_connections 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7180b6670 0x7ff7180b8b80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff72c10de50 0x7ff72c19fd20 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff72c105340 0x7ff72c19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 --2- 172.21.15.16:0/3437192287 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff72c1049f0 0x7ff72c19a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:30.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 >> 172.21.15.16:0/3437192287 conn(0x7ff72c079750 msgr2=0x7ff72c107970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 shutdown_connections 2024-09-21T02:59:30.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:30.284+0000 7ff733df5640 1 -- 172.21.15.16:0/3437192287 wait complete. 2024-09-21T02:59:30.878 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":293,"stamp":"2024-09-21T02:59:29.158504+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658357,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":194,"num_read_kb":193,"num_write":208,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386116,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555708,"statfs":{"total":1151940427776,"available":1151545044992,"internally_reserved":0,"allocated":12128256,"data_stored":6926199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":10,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003711"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737962+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:58:54.737962+0000","last_peered":"2024-09-21T02:58:54.737962+0000","last_clean":"2024-09-21T02:58:54.737962+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:58:54.737962+0000","last_undegraded":"2024-09-21T02:58:54.737962+0000","last_fullsized":"2024-09-21T02:58:54.737962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741271+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:58:54.741271+0000","last_peered":"2024-09-21T02:58:54.741271+0000","last_clean":"2024-09-21T02:58:54.741271+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:58:54.741271+0000","last_undegraded":"2024-09-21T02:58:54.741271+0000","last_fullsized":"2024-09-21T02:58:54.741271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739914+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:58:54.739914+0000","last_peered":"2024-09-21T02:58:54.739914+0000","last_clean":"2024-09-21T02:58:54.739914+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:58:54.739914+0000","last_undegraded":"2024-09-21T02:58:54.739914+0000","last_fullsized":"2024-09-21T02:58:54.739914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909663+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:58:54.909663+0000","last_peered":"2024-09-21T02:58:54.909663+0000","last_clean":"2024-09-21T02:58:54.909663+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:58:54.909663+0000","last_undegraded":"2024-09-21T02:58:54.909663+0000","last_fullsized":"2024-09-21T02:58:54.909663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.742957+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:58:54.742957+0000","last_peered":"2024-09-21T02:58:54.742957+0000","last_clean":"2024-09-21T02:58:54.742957+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:58:54.742957+0000","last_undegraded":"2024-09-21T02:58:54.742957+0000","last_fullsized":"2024-09-21T02:58:54.742957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737256+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:58:54.737256+0000","last_peered":"2024-09-21T02:58:54.737256+0000","last_clean":"2024-09-21T02:58:54.737256+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:58:54.737256+0000","last_undegraded":"2024-09-21T02:58:54.737256+0000","last_fullsized":"2024-09-21T02:58:54.737256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740196+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:58:54.740196+0000","last_peered":"2024-09-21T02:58:54.740196+0000","last_clean":"2024-09-21T02:58:54.740196+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:58:54.740196+0000","last_undegraded":"2024-09-21T02:58:54.740196+0000","last_fullsized":"2024-09-21T02:58:54.740196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736328+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:58:54.736328+0000","last_peered":"2024-09-21T02:58:54.736328+0000","last_clean":"2024-09-21T02:58:54.736328+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:58:54.736328+0000","last_undegraded":"2024-09-21T02:58:54.736328+0000","last_fullsized":"2024-09-21T02:58:54.736328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910336+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:58:54.910336+0000","last_peered":"2024-09-21T02:58:54.910336+0000","last_clean":"2024-09-21T02:58:54.910336+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:58:54.910336+0000","last_undegraded":"2024-09-21T02:58:54.910336+0000","last_fullsized":"2024-09-21T02:58:54.910336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737931+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:58:54.737931+0000","last_peered":"2024-09-21T02:58:54.737931+0000","last_clean":"2024-09-21T02:58:54.737931+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:58:54.737931+0000","last_undegraded":"2024-09-21T02:58:54.737931+0000","last_fullsized":"2024-09-21T02:58:54.737931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740813+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:58:54.740813+0000","last_peered":"2024-09-21T02:58:54.740813+0000","last_clean":"2024-09-21T02:58:54.740813+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:58:54.740813+0000","last_undegraded":"2024-09-21T02:58:54.740813+0000","last_fullsized":"2024-09-21T02:58:54.740813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909896+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:58:54.909896+0000","last_peered":"2024-09-21T02:58:54.909896+0000","last_clean":"2024-09-21T02:58:54.909896+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:58:54.909896+0000","last_undegraded":"2024-09-21T02:58:54.909896+0000","last_fullsized":"2024-09-21T02:58:54.909896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741666+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:58:54.741666+0000","last_peered":"2024-09-21T02:58:54.741666+0000","last_clean":"2024-09-21T02:58:54.741666+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:58:54.741666+0000","last_undegraded":"2024-09-21T02:58:54.741666+0000","last_fullsized":"2024-09-21T02:58:54.741666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738885+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:58:54.738885+0000","last_peered":"2024-09-21T02:58:54.738885+0000","last_clean":"2024-09-21T02:58:54.738885+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:58:54.738885+0000","last_undegraded":"2024-09-21T02:58:54.738885+0000","last_fullsized":"2024-09-21T02:58:54.738885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738662+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:58:54.738662+0000","last_peered":"2024-09-21T02:58:54.738662+0000","last_clean":"2024-09-21T02:58:54.738662+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:58:54.738662+0000","last_undegraded":"2024-09-21T02:58:54.738662+0000","last_fullsized":"2024-09-21T02:58:54.738662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.908742+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:58:54.908742+0000","last_peered":"2024-09-21T02:58:54.908742+0000","last_clean":"2024-09-21T02:58:54.908742+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:58:54.908742+0000","last_undegraded":"2024-09-21T02:58:54.908742+0000","last_fullsized":"2024-09-21T02:58:54.908742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737991+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:58:54.737991+0000","last_peered":"2024-09-21T02:58:54.737991+0000","last_clean":"2024-09-21T02:58:54.737991+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:58:54.737991+0000","last_undegraded":"2024-09-21T02:58:54.737991+0000","last_fullsized":"2024-09-21T02:58:54.737991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739745+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:58:54.739745+0000","last_peered":"2024-09-21T02:58:54.739745+0000","last_clean":"2024-09-21T02:58:54.739745+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:58:54.739745+0000","last_undegraded":"2024-09-21T02:58:54.739745+0000","last_fullsized":"2024-09-21T02:58:54.739745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741347+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:58:54.741347+0000","last_peered":"2024-09-21T02:58:54.741347+0000","last_clean":"2024-09-21T02:58:54.741347+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:58:54.741347+0000","last_undegraded":"2024-09-21T02:58:54.741347+0000","last_fullsized":"2024-09-21T02:58:54.741347+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737548+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:58:54.737548+0000","last_peered":"2024-09-21T02:58:54.737548+0000","last_clean":"2024-09-21T02:58:54.737548+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:58:54.737548+0000","last_undegraded":"2024-09-21T02:58:54.737548+0000","last_fullsized":"2024-09-21T02:58:54.737548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741546+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:58:54.741546+0000","last_peered":"2024-09-21T02:58:54.741546+0000","last_clean":"2024-09-21T02:58:54.741546+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:58:54.741546+0000","last_undegraded":"2024-09-21T02:58:54.741546+0000","last_fullsized":"2024-09-21T02:58:54.741546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909482+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:58:54.909482+0000","last_peered":"2024-09-21T02:58:54.909482+0000","last_clean":"2024-09-21T02:58:54.909482+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:58:54.909482+0000","last_undegraded":"2024-09-21T02:58:54.909482+0000","last_fullsized":"2024-09-21T02:58:54.909482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736385+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:58:54.736385+0000","last_peered":"2024-09-21T02:58:54.736385+0000","last_clean":"2024-09-21T02:58:54.736385+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:58:54.736385+0000","last_undegraded":"2024-09-21T02:58:54.736385+0000","last_fullsized":"2024-09-21T02:58:54.736385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.745373+0000","last_change":"2024-09-21T02:58:54.745262+0000","last_active":"2024-09-21T02:58:54.745373+0000","last_peered":"2024-09-21T02:58:54.745373+0000","last_clean":"2024-09-21T02:58:54.745373+0000","last_became_active":"2024-09-21T02:58:54.744809+0000","last_became_peered":"2024-09-21T02:58:54.744809+0000","last_unstale":"2024-09-21T02:58:54.745373+0000","last_undegraded":"2024-09-21T02:58:54.745373+0000","last_fullsized":"2024-09-21T02:58:54.745373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:32.627252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736774+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:58:54.736774+0000","last_peered":"2024-09-21T02:58:54.736774+0000","last_clean":"2024-09-21T02:58:54.736774+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:58:54.736774+0000","last_undegraded":"2024-09-21T02:58:54.736774+0000","last_fullsized":"2024-09-21T02:58:54.736774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741462+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:58:54.741462+0000","last_peered":"2024-09-21T02:58:54.741462+0000","last_clean":"2024-09-21T02:58:54.741462+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:58:54.741462+0000","last_undegraded":"2024-09-21T02:58:54.741462+0000","last_fullsized":"2024-09-21T02:58:54.741462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737755+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:58:54.737755+0000","last_peered":"2024-09-21T02:58:54.737755+0000","last_clean":"2024-09-21T02:58:54.737755+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:58:54.737755+0000","last_undegraded":"2024-09-21T02:58:54.737755+0000","last_fullsized":"2024-09-21T02:58:54.737755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744230+0000","last_change":"2024-09-21T02:58:54.744120+0000","last_active":"2024-09-21T02:58:54.744230+0000","last_peered":"2024-09-21T02:58:54.744230+0000","last_clean":"2024-09-21T02:58:54.744230+0000","last_became_active":"2024-09-21T02:58:54.743669+0000","last_became_peered":"2024-09-21T02:58:54.743669+0000","last_unstale":"2024-09-21T02:58:54.744230+0000","last_undegraded":"2024-09-21T02:58:54.744230+0000","last_fullsized":"2024-09-21T02:58:54.744230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:16.956302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739135+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:58:54.739135+0000","last_peered":"2024-09-21T02:58:54.739135+0000","last_clean":"2024-09-21T02:58:54.739135+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:58:54.739135+0000","last_undegraded":"2024-09-21T02:58:54.739135+0000","last_fullsized":"2024-09-21T02:58:54.739135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740581+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:58:54.740581+0000","last_peered":"2024-09-21T02:58:54.740581+0000","last_clean":"2024-09-21T02:58:54.740581+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:58:54.740581+0000","last_undegraded":"2024-09-21T02:58:54.740581+0000","last_fullsized":"2024-09-21T02:58:54.740581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740977+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:58:54.740977+0000","last_peered":"2024-09-21T02:58:54.740977+0000","last_clean":"2024-09-21T02:58:54.740977+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:58:54.740977+0000","last_undegraded":"2024-09-21T02:58:54.740977+0000","last_fullsized":"2024-09-21T02:58:54.740977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737477+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:58:54.737477+0000","last_peered":"2024-09-21T02:58:54.737477+0000","last_clean":"2024-09-21T02:58:54.737477+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:58:54.737477+0000","last_undegraded":"2024-09-21T02:58:54.737477+0000","last_fullsized":"2024-09-21T02:58:54.737477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743480+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:58:54.743480+0000","last_peered":"2024-09-21T02:58:54.743480+0000","last_clean":"2024-09-21T02:58:54.743480+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:58:54.743480+0000","last_undegraded":"2024-09-21T02:58:54.743480+0000","last_fullsized":"2024-09-21T02:58:54.743480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739058+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:58:54.739058+0000","last_peered":"2024-09-21T02:58:54.739058+0000","last_clean":"2024-09-21T02:58:54.739058+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:58:54.739058+0000","last_undegraded":"2024-09-21T02:58:54.739058+0000","last_fullsized":"2024-09-21T02:58:54.739058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738854+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:58:54.738854+0000","last_peered":"2024-09-21T02:58:54.738854+0000","last_clean":"2024-09-21T02:58:54.738854+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:58:54.738854+0000","last_undegraded":"2024-09-21T02:58:54.738854+0000","last_fullsized":"2024-09-21T02:58:54.738854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.912216+0000","last_change":"2024-09-21T02:58:54.912079+0000","last_active":"2024-09-21T02:58:54.912216+0000","last_peered":"2024-09-21T02:58:54.912216+0000","last_clean":"2024-09-21T02:58:54.912216+0000","last_became_active":"2024-09-21T02:58:54.911451+0000","last_became_peered":"2024-09-21T02:58:54.911451+0000","last_unstale":"2024-09-21T02:58:54.912216+0000","last_undegraded":"2024-09-21T02:58:54.912216+0000","last_fullsized":"2024-09-21T02:58:54.912216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:02.692841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740743+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:58:54.740743+0000","last_peered":"2024-09-21T02:58:54.740743+0000","last_clean":"2024-09-21T02:58:54.740743+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:58:54.740743+0000","last_undegraded":"2024-09-21T02:58:54.740743+0000","last_fullsized":"2024-09-21T02:58:54.740743+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736495+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:58:54.736495+0000","last_peered":"2024-09-21T02:58:54.736495+0000","last_clean":"2024-09-21T02:58:54.736495+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:58:54.736495+0000","last_undegraded":"2024-09-21T02:58:54.736495+0000","last_fullsized":"2024-09-21T02:58:54.736495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739416+0000","last_change":"2024-09-21T02:58:25.640688+0000","last_active":"2024-09-21T02:58:54.739416+0000","last_peered":"2024-09-21T02:58:54.739416+0000","last_clean":"2024-09-21T02:58:54.739416+0000","last_became_active":"2024-09-21T02:58:25.640306+0000","last_became_peered":"2024-09-21T02:58:25.640306+0000","last_unstale":"2024-09-21T02:58:54.739416+0000","last_undegraded":"2024-09-21T02:58:54.739416+0000","last_fullsized":"2024-09-21T02:58:54.739416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:04.808100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739431+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:58:54.739431+0000","last_peered":"2024-09-21T02:58:54.739431+0000","last_clean":"2024-09-21T02:58:54.739431+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:58:54.739431+0000","last_undegraded":"2024-09-21T02:58:54.739431+0000","last_fullsized":"2024-09-21T02:58:54.739431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737810+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:58:54.737810+0000","last_peered":"2024-09-21T02:58:54.737810+0000","last_clean":"2024-09-21T02:58:54.737810+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:58:54.737810+0000","last_undegraded":"2024-09-21T02:58:54.737810+0000","last_fullsized":"2024-09-21T02:58:54.737810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910149+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:58:54.910149+0000","last_peered":"2024-09-21T02:58:54.910149+0000","last_clean":"2024-09-21T02:58:54.910149+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:58:54.910149+0000","last_undegraded":"2024-09-21T02:58:54.910149+0000","last_fullsized":"2024-09-21T02:58:54.910149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740959+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:58:54.740959+0000","last_peered":"2024-09-21T02:58:54.740959+0000","last_clean":"2024-09-21T02:58:54.740959+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:58:54.740959+0000","last_undegraded":"2024-09-21T02:58:54.740959+0000","last_fullsized":"2024-09-21T02:58:54.740959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.745404+0000","last_change":"2024-09-21T02:58:54.745298+0000","last_active":"2024-09-21T02:58:54.745404+0000","last_peered":"2024-09-21T02:58:54.745404+0000","last_clean":"2024-09-21T02:58:54.745404+0000","last_became_active":"2024-09-21T02:58:54.744775+0000","last_became_peered":"2024-09-21T02:58:54.744775+0000","last_unstale":"2024-09-21T02:58:54.745404+0000","last_undegraded":"2024-09-21T02:58:54.745404+0000","last_fullsized":"2024-09-21T02:58:54.745404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:14.405920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739424+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:58:54.739424+0000","last_peered":"2024-09-21T02:58:54.739424+0000","last_clean":"2024-09-21T02:58:54.739424+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:58:54.739424+0000","last_undegraded":"2024-09-21T02:58:54.739424+0000","last_fullsized":"2024-09-21T02:58:54.739424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909747+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:58:54.909747+0000","last_peered":"2024-09-21T02:58:54.909747+0000","last_clean":"2024-09-21T02:58:54.909747+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:58:54.909747+0000","last_undegraded":"2024-09-21T02:58:54.909747+0000","last_fullsized":"2024-09-21T02:58:54.909747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'27","reported_seq":157,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:59:23.764586+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T02:59:23.764586+0000","last_peered":"2024-09-21T02:59:23.764586+0000","last_clean":"2024-09-21T02:59:23.764586+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T02:59:23.764586+0000","last_undegraded":"2024-09-21T02:59:23.764586+0000","last_fullsized":"2024-09-21T02:59:23.764586+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66957,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":107,"num_read_kb":118,"num_write":27,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743238+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:58:54.743238+0000","last_peered":"2024-09-21T02:58:54.743238+0000","last_clean":"2024-09-21T02:58:54.743238+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:58:54.743238+0000","last_undegraded":"2024-09-21T02:58:54.743238+0000","last_fullsized":"2024-09-21T02:58:54.743238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741205+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:58:54.741205+0000","last_peered":"2024-09-21T02:58:54.741205+0000","last_clean":"2024-09-21T02:58:54.741205+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:58:54.741205+0000","last_undegraded":"2024-09-21T02:58:54.741205+0000","last_fullsized":"2024-09-21T02:58:54.741205+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741073+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:58:54.741073+0000","last_peered":"2024-09-21T02:58:54.741073+0000","last_clean":"2024-09-21T02:58:54.741073+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:58:54.741073+0000","last_undegraded":"2024-09-21T02:58:54.741073+0000","last_fullsized":"2024-09-21T02:58:54.741073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738747+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:58:54.738747+0000","last_peered":"2024-09-21T02:58:54.738747+0000","last_clean":"2024-09-21T02:58:54.738747+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:58:54.738747+0000","last_undegraded":"2024-09-21T02:58:54.738747+0000","last_fullsized":"2024-09-21T02:58:54.738747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740670+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:58:54.740670+0000","last_peered":"2024-09-21T02:58:54.740670+0000","last_clean":"2024-09-21T02:58:54.740670+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:58:54.740670+0000","last_undegraded":"2024-09-21T02:58:54.740670+0000","last_fullsized":"2024-09-21T02:58:54.740670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909289+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:58:54.909289+0000","last_peered":"2024-09-21T02:58:54.909289+0000","last_clean":"2024-09-21T02:58:54.909289+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:58:54.909289+0000","last_undegraded":"2024-09-21T02:58:54.909289+0000","last_fullsized":"2024-09-21T02:58:54.909289+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740143+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:58:54.740143+0000","last_peered":"2024-09-21T02:58:54.740143+0000","last_clean":"2024-09-21T02:58:54.740143+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:58:54.740143+0000","last_undegraded":"2024-09-21T02:58:54.740143+0000","last_fullsized":"2024-09-21T02:58:54.740143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.742929+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:58:54.742929+0000","last_peered":"2024-09-21T02:58:54.742929+0000","last_clean":"2024-09-21T02:58:54.742929+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:58:54.742929+0000","last_undegraded":"2024-09-21T02:58:54.742929+0000","last_fullsized":"2024-09-21T02:58:54.742929+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738601+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:58:54.738601+0000","last_peered":"2024-09-21T02:58:54.738601+0000","last_clean":"2024-09-21T02:58:54.738601+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:58:54.738601+0000","last_undegraded":"2024-09-21T02:58:54.738601+0000","last_fullsized":"2024-09-21T02:58:54.738601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743275+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:58:54.743275+0000","last_peered":"2024-09-21T02:58:54.743275+0000","last_clean":"2024-09-21T02:58:54.743275+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:58:54.743275+0000","last_undegraded":"2024-09-21T02:58:54.743275+0000","last_fullsized":"2024-09-21T02:58:54.743275+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737834+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:58:54.737834+0000","last_peered":"2024-09-21T02:58:54.737834+0000","last_clean":"2024-09-21T02:58:54.737834+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:58:54.737834+0000","last_undegraded":"2024-09-21T02:58:54.737834+0000","last_fullsized":"2024-09-21T02:58:54.737834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741546+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:58:54.741546+0000","last_peered":"2024-09-21T02:58:54.741546+0000","last_clean":"2024-09-21T02:58:54.741546+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:58:54.741546+0000","last_undegraded":"2024-09-21T02:58:54.741546+0000","last_fullsized":"2024-09-21T02:58:54.741546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740954+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:58:54.740954+0000","last_peered":"2024-09-21T02:58:54.740954+0000","last_clean":"2024-09-21T02:58:54.740954+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:58:54.740954+0000","last_undegraded":"2024-09-21T02:58:54.740954+0000","last_fullsized":"2024-09-21T02:58:54.740954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739462+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:58:54.739462+0000","last_peered":"2024-09-21T02:58:54.739462+0000","last_clean":"2024-09-21T02:58:54.739462+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:58:54.739462+0000","last_undegraded":"2024-09-21T02:58:54.739462+0000","last_fullsized":"2024-09-21T02:58:54.739462+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740785+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:58:54.740785+0000","last_peered":"2024-09-21T02:58:54.740785+0000","last_clean":"2024-09-21T02:58:54.740785+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:58:54.740785+0000","last_undegraded":"2024-09-21T02:58:54.740785+0000","last_fullsized":"2024-09-21T02:58:54.740785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741748+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:58:54.741748+0000","last_peered":"2024-09-21T02:58:54.741748+0000","last_clean":"2024-09-21T02:58:54.741748+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:58:54.741748+0000","last_undegraded":"2024-09-21T02:58:54.741748+0000","last_fullsized":"2024-09-21T02:58:54.741748+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740858+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:58:54.740858+0000","last_peered":"2024-09-21T02:58:54.740858+0000","last_clean":"2024-09-21T02:58:54.740858+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:58:54.740858+0000","last_undegraded":"2024-09-21T02:58:54.740858+0000","last_fullsized":"2024-09-21T02:58:54.740858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740399+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:58:54.740399+0000","last_peered":"2024-09-21T02:58:54.740399+0000","last_clean":"2024-09-21T02:58:54.740399+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:58:54.740399+0000","last_undegraded":"2024-09-21T02:58:54.740399+0000","last_fullsized":"2024-09-21T02:58:54.740399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909443+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:58:54.909443+0000","last_peered":"2024-09-21T02:58:54.909443+0000","last_clean":"2024-09-21T02:58:54.909443+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:58:54.909443+0000","last_undegraded":"2024-09-21T02:58:54.909443+0000","last_fullsized":"2024-09-21T02:58:54.909443+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736591+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:58:54.736591+0000","last_peered":"2024-09-21T02:58:54.736591+0000","last_clean":"2024-09-21T02:58:54.736591+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:58:54.736591+0000","last_undegraded":"2024-09-21T02:58:54.736591+0000","last_fullsized":"2024-09-21T02:58:54.736591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740396+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:58:54.740396+0000","last_peered":"2024-09-21T02:58:54.740396+0000","last_clean":"2024-09-21T02:58:54.740396+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:58:54.740396+0000","last_undegraded":"2024-09-21T02:58:54.740396+0000","last_fullsized":"2024-09-21T02:58:54.740396+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740622+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:54.740622+0000","last_peered":"2024-09-21T02:58:54.740622+0000","last_clean":"2024-09-21T02:58:54.740622+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:58:54.740622+0000","last_undegraded":"2024-09-21T02:58:54.740622+0000","last_fullsized":"2024-09-21T02:58:54.740622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740908+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:58:54.740908+0000","last_peered":"2024-09-21T02:58:54.740908+0000","last_clean":"2024-09-21T02:58:54.740908+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:58:54.740908+0000","last_undegraded":"2024-09-21T02:58:54.740908+0000","last_fullsized":"2024-09-21T02:58:54.740908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909981+0000","last_change":"2024-09-21T02:58:54.909889+0000","last_active":"2024-09-21T02:58:54.909981+0000","last_peered":"2024-09-21T02:58:54.909981+0000","last_clean":"2024-09-21T02:58:54.909981+0000","last_became_active":"2024-09-21T02:58:54.909607+0000","last_became_peered":"2024-09-21T02:58:54.909607+0000","last_unstale":"2024-09-21T02:58:54.909981+0000","last_undegraded":"2024-09-21T02:58:54.909981+0000","last_fullsized":"2024-09-21T02:58:54.909981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740639+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:58:54.740639+0000","last_peered":"2024-09-21T02:58:54.740639+0000","last_clean":"2024-09-21T02:58:54.740639+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:58:54.740639+0000","last_undegraded":"2024-09-21T02:58:54.740639+0000","last_fullsized":"2024-09-21T02:58:54.740639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737415+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:58:54.737415+0000","last_peered":"2024-09-21T02:58:54.737415+0000","last_clean":"2024-09-21T02:58:54.737415+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:58:54.737415+0000","last_undegraded":"2024-09-21T02:58:54.737415+0000","last_fullsized":"2024-09-21T02:58:54.737415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739708+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:58:54.739708+0000","last_peered":"2024-09-21T02:58:54.739708+0000","last_clean":"2024-09-21T02:58:54.739708+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:58:54.739708+0000","last_undegraded":"2024-09-21T02:58:54.739708+0000","last_fullsized":"2024-09-21T02:58:54.739708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737407+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:58:54.737407+0000","last_peered":"2024-09-21T02:58:54.737407+0000","last_clean":"2024-09-21T02:58:54.737407+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:58:54.737407+0000","last_undegraded":"2024-09-21T02:58:54.737407+0000","last_fullsized":"2024-09-21T02:58:54.737407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744125+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:58:54.744125+0000","last_peered":"2024-09-21T02:58:54.744125+0000","last_clean":"2024-09-21T02:58:54.744125+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:58:54.744125+0000","last_undegraded":"2024-09-21T02:58:54.744125+0000","last_fullsized":"2024-09-21T02:58:54.744125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741141+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:58:54.741141+0000","last_peered":"2024-09-21T02:58:54.741141+0000","last_clean":"2024-09-21T02:58:54.741141+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:58:54.741141+0000","last_undegraded":"2024-09-21T02:58:54.741141+0000","last_fullsized":"2024-09-21T02:58:54.741141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739911+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:58:54.739911+0000","last_peered":"2024-09-21T02:58:54.739911+0000","last_clean":"2024-09-21T02:58:54.739911+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:58:54.739911+0000","last_undegraded":"2024-09-21T02:58:54.739911+0000","last_fullsized":"2024-09-21T02:58:54.739911+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736819+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:58:54.736819+0000","last_peered":"2024-09-21T02:58:54.736819+0000","last_clean":"2024-09-21T02:58:54.736819+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:58:54.736819+0000","last_undegraded":"2024-09-21T02:58:54.736819+0000","last_fullsized":"2024-09-21T02:58:54.736819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910580+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:58:54.910580+0000","last_peered":"2024-09-21T02:58:54.910580+0000","last_clean":"2024-09-21T02:58:54.910580+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:58:54.910580+0000","last_undegraded":"2024-09-21T02:58:54.910580+0000","last_fullsized":"2024-09-21T02:58:54.910580+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743644+0000","last_change":"2024-09-21T02:58:54.743546+0000","last_active":"2024-09-21T02:58:54.743644+0000","last_peered":"2024-09-21T02:58:54.743644+0000","last_clean":"2024-09-21T02:58:54.743644+0000","last_became_active":"2024-09-21T02:58:54.743107+0000","last_became_peered":"2024-09-21T02:58:54.743107+0000","last_unstale":"2024-09-21T02:58:54.743644+0000","last_undegraded":"2024-09-21T02:58:54.743644+0000","last_fullsized":"2024-09-21T02:58:54.743644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.458721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909028+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:58:54.909028+0000","last_peered":"2024-09-21T02:58:54.909028+0000","last_clean":"2024-09-21T02:58:54.909028+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:58:54.909028+0000","last_undegraded":"2024-09-21T02:58:54.909028+0000","last_fullsized":"2024-09-21T02:58:54.909028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743822+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:58:54.743822+0000","last_peered":"2024-09-21T02:58:54.743822+0000","last_clean":"2024-09-21T02:58:54.743822+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:58:54.743822+0000","last_undegraded":"2024-09-21T02:58:54.743822+0000","last_fullsized":"2024-09-21T02:58:54.743822+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736508+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:58:54.736508+0000","last_peered":"2024-09-21T02:58:54.736508+0000","last_clean":"2024-09-21T02:58:54.736508+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:58:54.736508+0000","last_undegraded":"2024-09-21T02:58:54.736508+0000","last_fullsized":"2024-09-21T02:58:54.736508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737700+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:58:54.737700+0000","last_peered":"2024-09-21T02:58:54.737700+0000","last_clean":"2024-09-21T02:58:54.737700+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:58:54.737700+0000","last_undegraded":"2024-09-21T02:58:54.737700+0000","last_fullsized":"2024-09-21T02:58:54.737700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740708+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:58:54.740708+0000","last_peered":"2024-09-21T02:58:54.740708+0000","last_clean":"2024-09-21T02:58:54.740708+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:58:54.740708+0000","last_undegraded":"2024-09-21T02:58:54.740708+0000","last_fullsized":"2024-09-21T02:58:54.740708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736662+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:58:54.736662+0000","last_peered":"2024-09-21T02:58:54.736662+0000","last_clean":"2024-09-21T02:58:54.736662+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:58:54.736662+0000","last_undegraded":"2024-09-21T02:58:54.736662+0000","last_fullsized":"2024-09-21T02:58:54.736662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.914626+0000","last_change":"2024-09-21T02:58:54.914527+0000","last_active":"2024-09-21T02:58:54.914626+0000","last_peered":"2024-09-21T02:58:54.914626+0000","last_clean":"2024-09-21T02:58:54.914626+0000","last_became_active":"2024-09-21T02:58:54.910405+0000","last_became_peered":"2024-09-21T02:58:54.910405+0000","last_unstale":"2024-09-21T02:58:54.914626+0000","last_undegraded":"2024-09-21T02:58:54.914626+0000","last_fullsized":"2024-09-21T02:58:54.914626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:50.419816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740322+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:58:54.740322+0000","last_peered":"2024-09-21T02:58:54.740322+0000","last_clean":"2024-09-21T02:58:54.740322+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:58:54.740322+0000","last_undegraded":"2024-09-21T02:58:54.740322+0000","last_fullsized":"2024-09-21T02:58:54.740322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739187+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:58:54.739187+0000","last_peered":"2024-09-21T02:58:54.739187+0000","last_clean":"2024-09-21T02:58:54.739187+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:58:54.739187+0000","last_undegraded":"2024-09-21T02:58:54.739187+0000","last_fullsized":"2024-09-21T02:58:54.739187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743489+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:58:54.743489+0000","last_peered":"2024-09-21T02:58:54.743489+0000","last_clean":"2024-09-21T02:58:54.743489+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:58:54.743489+0000","last_undegraded":"2024-09-21T02:58:54.743489+0000","last_fullsized":"2024-09-21T02:58:54.743489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909737+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:58:54.909737+0000","last_peered":"2024-09-21T02:58:54.909737+0000","last_clean":"2024-09-21T02:58:54.909737+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:58:54.909737+0000","last_undegraded":"2024-09-21T02:58:54.909737+0000","last_fullsized":"2024-09-21T02:58:54.909737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.918634+0000","last_change":"2024-09-21T02:58:54.918104+0000","last_active":"2024-09-21T02:58:54.918634+0000","last_peered":"2024-09-21T02:58:54.918634+0000","last_clean":"2024-09-21T02:58:54.918634+0000","last_became_active":"2024-09-21T02:58:54.911376+0000","last_became_peered":"2024-09-21T02:58:54.911376+0000","last_unstale":"2024-09-21T02:58:54.918634+0000","last_undegraded":"2024-09-21T02:58:54.918634+0000","last_fullsized":"2024-09-21T02:58:54.918634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:42.992181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740360+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:58:54.740360+0000","last_peered":"2024-09-21T02:58:54.740360+0000","last_clean":"2024-09-21T02:58:54.740360+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:58:54.740360+0000","last_undegraded":"2024-09-21T02:58:54.740360+0000","last_fullsized":"2024-09-21T02:58:54.740360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737093+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:58:54.737093+0000","last_peered":"2024-09-21T02:58:54.737093+0000","last_clean":"2024-09-21T02:58:54.737093+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:58:54.737093+0000","last_undegraded":"2024-09-21T02:58:54.737093+0000","last_fullsized":"2024-09-21T02:58:54.737093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744135+0000","last_change":"2024-09-21T02:58:54.743356+0000","last_active":"2024-09-21T02:58:54.744135+0000","last_peered":"2024-09-21T02:58:54.744135+0000","last_clean":"2024-09-21T02:58:54.744135+0000","last_became_active":"2024-09-21T02:58:54.742875+0000","last_became_peered":"2024-09-21T02:58:54.742875+0000","last_unstale":"2024-09-21T02:58:54.744135+0000","last_undegraded":"2024-09-21T02:58:54.744135+0000","last_fullsized":"2024-09-21T02:58:54.744135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:47.202904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739505+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:58:54.739505+0000","last_peered":"2024-09-21T02:58:54.739505+0000","last_clean":"2024-09-21T02:58:54.739505+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:58:54.739505+0000","last_undegraded":"2024-09-21T02:58:54.739505+0000","last_fullsized":"2024-09-21T02:58:54.739505+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67755,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":113,"num_read_kb":124,"num_write":83,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":203265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":412594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906974,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40992,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704160,"statfs":{"total":95995035648,"available":95953059840,"internally_reserved":0,"allocated":1523712,"data_stored":1070230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070499,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":413176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234024,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":843776,"data_stored":413198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430252,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1437696,"data_stored":1003661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593778,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":412711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757302,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1445888,"data_stored":1003261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953531,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":412616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":412932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378695,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":479585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":479551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":412684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:59:30.882 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph pg dump --format=json 2024-09-21T02:59:31.161 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:30 smithi018 ceph-mon[34587]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:30 smithi123 ceph-mon[34645]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:30 smithi016 ceph-mon[31253]: pgmap v293: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 -- 172.21.15.16:0/570593477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0108cf0 msgr2=0x7f19a010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 --2- 172.21.15.16:0/570593477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0108cf0 0x7f19a010f520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f1998009ca0 tx=0x7f199802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 -- 172.21.15.16:0/570593477 shutdown_connections 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 --2- 172.21.15.16:0/570593477 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0108cf0 0x7f19a010f520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 --2- 172.21.15.16:0/570593477 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a01083a0 0x7f19a01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 --2- 172.21.15.16:0/570593477 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a0102a60 0x7f19a0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 -- 172.21.15.16:0/570593477 >> 172.21.15.16:0/570593477 conn(0x7f19a0079730 msgr2=0x7f19a0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 -- 172.21.15.16:0/570593477 shutdown_connections 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.632+0000 7f19a781f640 1 -- 172.21.15.16:0/570593477 wait complete. 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 Processor -- start 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 -- start start 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a01083a0 0x7f19a00ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 0x7f19a01000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0112960 con 0x7f19a0102a60 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f19a01127e0 con 0x7f19a01083a0 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a781f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f19a0112ae0 con 0x7f19a0108cf0 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a55bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a55bb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57772/0 (socket says 172.21.15.16:57772) 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a55bb640 1 -- 172.21.15.16:0/971725121 learned_addr learned my addr 172.21.15.16:0/971725121 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.633+0000 7f19a4dba640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a01083a0 0x7f19a00ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a5dbc640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 0x7f19a01000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 -- 172.21.15.16:0/971725121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 msgr2=0x7f19a01000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 0x7f19a01000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 -- 172.21.15.16:0/971725121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a01083a0 msgr2=0x7f19a00ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a01083a0 0x7f19a00ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 -- 172.21.15.16:0/971725121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19a01a8760 con 0x7f19a0102a60 2024-09-21T02:59:31.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a5dbc640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 0x7f19a01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a55bb640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f199c00d950 tx=0x7f199c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c017070 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19a01a8a50 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f199c009c30 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19a01a8fe0 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.634+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f199c00f620 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.635+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19a0102e70 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.637+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f199c005030 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.638+0000 7f19967fc640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 0x7f198c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.638+0000 7f19a4dba640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 0x7f198c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.638+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f199c0e71d0 con 0x7f19a0102a60 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.638+0000 7f19a4dba640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 0x7f198c0b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a006a1d0 tx=0x7f199000a250 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:31.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.641+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f199c0ad820 con 0x7f19a0102a60 2024-09-21T02:59:31.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.828+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f19a0063800 con 0x7f198c0b65f0 2024-09-21T02:59:31.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.832+0000 7f19967fc640 1 -- 172.21.15.16:0/971725121 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258617 (secure 0 0 0) 0x7f19a0063800 con 0x7f198c0b65f0 2024-09-21T02:59:31.836 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:31.837 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 msgr2=0x7f198c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 0x7f198c0b8b00 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f19a006a1d0 tx=0x7f199000a250 comp rx=0 tx=0).stop 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 msgr2=0x7f19a00ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f199c00d950 tx=0x7f199c00de10 comp rx=0 tx=0).stop 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 shutdown_connections 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f198c0b65f0 0x7f198c0b8b00 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19a0108cf0 0x7f19a01000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f19a01083a0 0x7f19a00ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 --2- 172.21.15.16:0/971725121 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f19a0102a60 0x7f19a00ff670 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:31.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.835+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 >> 172.21.15.16:0/971725121 conn(0x7f19a0079730 msgr2=0x7f19a006b200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:31.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.836+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 shutdown_connections 2024-09-21T02:59:31.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:31.836+0000 7f19a781f640 1 -- 172.21.15.16:0/971725121 wait complete. 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:32.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:31 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:32.212 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:31 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='client.15162 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:32.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T02:59:32.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T02:59:32.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:31 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T02:59:32.561 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":294,"stamp":"2024-09-21T02:59:31.158947+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658357,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":204,"num_read_kb":198,"num_write":208,"num_write_kb":2245,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":162,"ondisk_log_size":162,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386116,"kb_used_data":11844,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124555708,"statfs":{"total":1151940427776,"available":1151545044992,"internally_reserved":0,"allocated":12128256,"data_stored":6926199,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":382973241},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":30,"num_read_kb":15,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003501"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737962+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:58:54.737962+0000","last_peered":"2024-09-21T02:58:54.737962+0000","last_clean":"2024-09-21T02:58:54.737962+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:58:54.737962+0000","last_undegraded":"2024-09-21T02:58:54.737962+0000","last_fullsized":"2024-09-21T02:58:54.737962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741271+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:58:54.741271+0000","last_peered":"2024-09-21T02:58:54.741271+0000","last_clean":"2024-09-21T02:58:54.741271+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:58:54.741271+0000","last_undegraded":"2024-09-21T02:58:54.741271+0000","last_fullsized":"2024-09-21T02:58:54.741271+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739914+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:58:54.739914+0000","last_peered":"2024-09-21T02:58:54.739914+0000","last_clean":"2024-09-21T02:58:54.739914+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:58:54.739914+0000","last_undegraded":"2024-09-21T02:58:54.739914+0000","last_fullsized":"2024-09-21T02:58:54.739914+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909663+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:58:54.909663+0000","last_peered":"2024-09-21T02:58:54.909663+0000","last_clean":"2024-09-21T02:58:54.909663+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:58:54.909663+0000","last_undegraded":"2024-09-21T02:58:54.909663+0000","last_fullsized":"2024-09-21T02:58:54.909663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.742957+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:58:54.742957+0000","last_peered":"2024-09-21T02:58:54.742957+0000","last_clean":"2024-09-21T02:58:54.742957+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:58:54.742957+0000","last_undegraded":"2024-09-21T02:58:54.742957+0000","last_fullsized":"2024-09-21T02:58:54.742957+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737256+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:58:54.737256+0000","last_peered":"2024-09-21T02:58:54.737256+0000","last_clean":"2024-09-21T02:58:54.737256+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:58:54.737256+0000","last_undegraded":"2024-09-21T02:58:54.737256+0000","last_fullsized":"2024-09-21T02:58:54.737256+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740196+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:58:54.740196+0000","last_peered":"2024-09-21T02:58:54.740196+0000","last_clean":"2024-09-21T02:58:54.740196+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:58:54.740196+0000","last_undegraded":"2024-09-21T02:58:54.740196+0000","last_fullsized":"2024-09-21T02:58:54.740196+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736328+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:58:54.736328+0000","last_peered":"2024-09-21T02:58:54.736328+0000","last_clean":"2024-09-21T02:58:54.736328+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:58:54.736328+0000","last_undegraded":"2024-09-21T02:58:54.736328+0000","last_fullsized":"2024-09-21T02:58:54.736328+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910336+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:58:54.910336+0000","last_peered":"2024-09-21T02:58:54.910336+0000","last_clean":"2024-09-21T02:58:54.910336+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:58:54.910336+0000","last_undegraded":"2024-09-21T02:58:54.910336+0000","last_fullsized":"2024-09-21T02:58:54.910336+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737931+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:58:54.737931+0000","last_peered":"2024-09-21T02:58:54.737931+0000","last_clean":"2024-09-21T02:58:54.737931+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:58:54.737931+0000","last_undegraded":"2024-09-21T02:58:54.737931+0000","last_fullsized":"2024-09-21T02:58:54.737931+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740813+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:58:54.740813+0000","last_peered":"2024-09-21T02:58:54.740813+0000","last_clean":"2024-09-21T02:58:54.740813+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:58:54.740813+0000","last_undegraded":"2024-09-21T02:58:54.740813+0000","last_fullsized":"2024-09-21T02:58:54.740813+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909896+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:58:54.909896+0000","last_peered":"2024-09-21T02:58:54.909896+0000","last_clean":"2024-09-21T02:58:54.909896+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:58:54.909896+0000","last_undegraded":"2024-09-21T02:58:54.909896+0000","last_fullsized":"2024-09-21T02:58:54.909896+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741666+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:58:54.741666+0000","last_peered":"2024-09-21T02:58:54.741666+0000","last_clean":"2024-09-21T02:58:54.741666+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:58:54.741666+0000","last_undegraded":"2024-09-21T02:58:54.741666+0000","last_fullsized":"2024-09-21T02:58:54.741666+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738885+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:58:54.738885+0000","last_peered":"2024-09-21T02:58:54.738885+0000","last_clean":"2024-09-21T02:58:54.738885+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:58:54.738885+0000","last_undegraded":"2024-09-21T02:58:54.738885+0000","last_fullsized":"2024-09-21T02:58:54.738885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738662+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:58:54.738662+0000","last_peered":"2024-09-21T02:58:54.738662+0000","last_clean":"2024-09-21T02:58:54.738662+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:58:54.738662+0000","last_undegraded":"2024-09-21T02:58:54.738662+0000","last_fullsized":"2024-09-21T02:58:54.738662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.908742+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:58:54.908742+0000","last_peered":"2024-09-21T02:58:54.908742+0000","last_clean":"2024-09-21T02:58:54.908742+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:58:54.908742+0000","last_undegraded":"2024-09-21T02:58:54.908742+0000","last_fullsized":"2024-09-21T02:58:54.908742+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737991+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:58:54.737991+0000","last_peered":"2024-09-21T02:58:54.737991+0000","last_clean":"2024-09-21T02:58:54.737991+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:58:54.737991+0000","last_undegraded":"2024-09-21T02:58:54.737991+0000","last_fullsized":"2024-09-21T02:58:54.737991+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739745+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:58:54.739745+0000","last_peered":"2024-09-21T02:58:54.739745+0000","last_clean":"2024-09-21T02:58:54.739745+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:58:54.739745+0000","last_undegraded":"2024-09-21T02:58:54.739745+0000","last_fullsized":"2024-09-21T02:58:54.739745+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741347+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:58:54.741347+0000","last_peered":"2024-09-21T02:58:54.741347+0000","last_clean":"2024-09-21T02:58:54.741347+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:58:54.741347+0000","last_undegraded":"2024-09-21T02:58:54.741347+0000","last_fullsized":"2024-09-21T02:58:54.741347+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737548+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:58:54.737548+0000","last_peered":"2024-09-21T02:58:54.737548+0000","last_clean":"2024-09-21T02:58:54.737548+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:58:54.737548+0000","last_undegraded":"2024-09-21T02:58:54.737548+0000","last_fullsized":"2024-09-21T02:58:54.737548+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741546+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:58:54.741546+0000","last_peered":"2024-09-21T02:58:54.741546+0000","last_clean":"2024-09-21T02:58:54.741546+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:58:54.741546+0000","last_undegraded":"2024-09-21T02:58:54.741546+0000","last_fullsized":"2024-09-21T02:58:54.741546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909482+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:58:54.909482+0000","last_peered":"2024-09-21T02:58:54.909482+0000","last_clean":"2024-09-21T02:58:54.909482+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:58:54.909482+0000","last_undegraded":"2024-09-21T02:58:54.909482+0000","last_fullsized":"2024-09-21T02:58:54.909482+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736385+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:58:54.736385+0000","last_peered":"2024-09-21T02:58:54.736385+0000","last_clean":"2024-09-21T02:58:54.736385+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:58:54.736385+0000","last_undegraded":"2024-09-21T02:58:54.736385+0000","last_fullsized":"2024-09-21T02:58:54.736385+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.745373+0000","last_change":"2024-09-21T02:58:54.745262+0000","last_active":"2024-09-21T02:58:54.745373+0000","last_peered":"2024-09-21T02:58:54.745373+0000","last_clean":"2024-09-21T02:58:54.745373+0000","last_became_active":"2024-09-21T02:58:54.744809+0000","last_became_peered":"2024-09-21T02:58:54.744809+0000","last_unstale":"2024-09-21T02:58:54.745373+0000","last_undegraded":"2024-09-21T02:58:54.745373+0000","last_fullsized":"2024-09-21T02:58:54.745373+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:32.627252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736774+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:58:54.736774+0000","last_peered":"2024-09-21T02:58:54.736774+0000","last_clean":"2024-09-21T02:58:54.736774+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:58:54.736774+0000","last_undegraded":"2024-09-21T02:58:54.736774+0000","last_fullsized":"2024-09-21T02:58:54.736774+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741462+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:58:54.741462+0000","last_peered":"2024-09-21T02:58:54.741462+0000","last_clean":"2024-09-21T02:58:54.741462+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:58:54.741462+0000","last_undegraded":"2024-09-21T02:58:54.741462+0000","last_fullsized":"2024-09-21T02:58:54.741462+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737755+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:58:54.737755+0000","last_peered":"2024-09-21T02:58:54.737755+0000","last_clean":"2024-09-21T02:58:54.737755+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:58:54.737755+0000","last_undegraded":"2024-09-21T02:58:54.737755+0000","last_fullsized":"2024-09-21T02:58:54.737755+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744230+0000","last_change":"2024-09-21T02:58:54.744120+0000","last_active":"2024-09-21T02:58:54.744230+0000","last_peered":"2024-09-21T02:58:54.744230+0000","last_clean":"2024-09-21T02:58:54.744230+0000","last_became_active":"2024-09-21T02:58:54.743669+0000","last_became_peered":"2024-09-21T02:58:54.743669+0000","last_unstale":"2024-09-21T02:58:54.744230+0000","last_undegraded":"2024-09-21T02:58:54.744230+0000","last_fullsized":"2024-09-21T02:58:54.744230+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:16.956302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739135+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:58:54.739135+0000","last_peered":"2024-09-21T02:58:54.739135+0000","last_clean":"2024-09-21T02:58:54.739135+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:58:54.739135+0000","last_undegraded":"2024-09-21T02:58:54.739135+0000","last_fullsized":"2024-09-21T02:58:54.739135+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740581+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:58:54.740581+0000","last_peered":"2024-09-21T02:58:54.740581+0000","last_clean":"2024-09-21T02:58:54.740581+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:58:54.740581+0000","last_undegraded":"2024-09-21T02:58:54.740581+0000","last_fullsized":"2024-09-21T02:58:54.740581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740977+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:58:54.740977+0000","last_peered":"2024-09-21T02:58:54.740977+0000","last_clean":"2024-09-21T02:58:54.740977+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:58:54.740977+0000","last_undegraded":"2024-09-21T02:58:54.740977+0000","last_fullsized":"2024-09-21T02:58:54.740977+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737477+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:58:54.737477+0000","last_peered":"2024-09-21T02:58:54.737477+0000","last_clean":"2024-09-21T02:58:54.737477+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:58:54.737477+0000","last_undegraded":"2024-09-21T02:58:54.737477+0000","last_fullsized":"2024-09-21T02:58:54.737477+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743480+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:58:54.743480+0000","last_peered":"2024-09-21T02:58:54.743480+0000","last_clean":"2024-09-21T02:58:54.743480+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:58:54.743480+0000","last_undegraded":"2024-09-21T02:58:54.743480+0000","last_fullsized":"2024-09-21T02:58:54.743480+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739058+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:58:54.739058+0000","last_peered":"2024-09-21T02:58:54.739058+0000","last_clean":"2024-09-21T02:58:54.739058+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:58:54.739058+0000","last_undegraded":"2024-09-21T02:58:54.739058+0000","last_fullsized":"2024-09-21T02:58:54.739058+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738854+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:58:54.738854+0000","last_peered":"2024-09-21T02:58:54.738854+0000","last_clean":"2024-09-21T02:58:54.738854+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:58:54.738854+0000","last_undegraded":"2024-09-21T02:58:54.738854+0000","last_fullsized":"2024-09-21T02:58:54.738854+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.912216+0000","last_change":"2024-09-21T02:58:54.912079+0000","last_active":"2024-09-21T02:58:54.912216+0000","last_peered":"2024-09-21T02:58:54.912216+0000","last_clean":"2024-09-21T02:58:54.912216+0000","last_became_active":"2024-09-21T02:58:54.911451+0000","last_became_peered":"2024-09-21T02:58:54.911451+0000","last_unstale":"2024-09-21T02:58:54.912216+0000","last_undegraded":"2024-09-21T02:58:54.912216+0000","last_fullsized":"2024-09-21T02:58:54.912216+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:02.692841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740743+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:58:54.740743+0000","last_peered":"2024-09-21T02:58:54.740743+0000","last_clean":"2024-09-21T02:58:54.740743+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:58:54.740743+0000","last_undegraded":"2024-09-21T02:58:54.740743+0000","last_fullsized":"2024-09-21T02:58:54.740743+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736495+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:58:54.736495+0000","last_peered":"2024-09-21T02:58:54.736495+0000","last_clean":"2024-09-21T02:58:54.736495+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:58:54.736495+0000","last_undegraded":"2024-09-21T02:58:54.736495+0000","last_fullsized":"2024-09-21T02:58:54.736495+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739416+0000","last_change":"2024-09-21T02:58:25.640688+0000","last_active":"2024-09-21T02:58:54.739416+0000","last_peered":"2024-09-21T02:58:54.739416+0000","last_clean":"2024-09-21T02:58:54.739416+0000","last_became_active":"2024-09-21T02:58:25.640306+0000","last_became_peered":"2024-09-21T02:58:25.640306+0000","last_unstale":"2024-09-21T02:58:54.739416+0000","last_undegraded":"2024-09-21T02:58:54.739416+0000","last_fullsized":"2024-09-21T02:58:54.739416+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:28:04.808100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739431+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:58:54.739431+0000","last_peered":"2024-09-21T02:58:54.739431+0000","last_clean":"2024-09-21T02:58:54.739431+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:58:54.739431+0000","last_undegraded":"2024-09-21T02:58:54.739431+0000","last_fullsized":"2024-09-21T02:58:54.739431+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737810+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:58:54.737810+0000","last_peered":"2024-09-21T02:58:54.737810+0000","last_clean":"2024-09-21T02:58:54.737810+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:58:54.737810+0000","last_undegraded":"2024-09-21T02:58:54.737810+0000","last_fullsized":"2024-09-21T02:58:54.737810+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910149+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:58:54.910149+0000","last_peered":"2024-09-21T02:58:54.910149+0000","last_clean":"2024-09-21T02:58:54.910149+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:58:54.910149+0000","last_undegraded":"2024-09-21T02:58:54.910149+0000","last_fullsized":"2024-09-21T02:58:54.910149+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740959+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:58:54.740959+0000","last_peered":"2024-09-21T02:58:54.740959+0000","last_clean":"2024-09-21T02:58:54.740959+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:58:54.740959+0000","last_undegraded":"2024-09-21T02:58:54.740959+0000","last_fullsized":"2024-09-21T02:58:54.740959+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.745404+0000","last_change":"2024-09-21T02:58:54.745298+0000","last_active":"2024-09-21T02:58:54.745404+0000","last_peered":"2024-09-21T02:58:54.745404+0000","last_clean":"2024-09-21T02:58:54.745404+0000","last_became_active":"2024-09-21T02:58:54.744775+0000","last_became_peered":"2024-09-21T02:58:54.744775+0000","last_unstale":"2024-09-21T02:58:54.745404+0000","last_undegraded":"2024-09-21T02:58:54.745404+0000","last_fullsized":"2024-09-21T02:58:54.745404+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:14.405920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739424+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:58:54.739424+0000","last_peered":"2024-09-21T02:58:54.739424+0000","last_clean":"2024-09-21T02:58:54.739424+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:58:54.739424+0000","last_undegraded":"2024-09-21T02:58:54.739424+0000","last_fullsized":"2024-09-21T02:58:54.739424+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909747+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:58:54.909747+0000","last_peered":"2024-09-21T02:58:54.909747+0000","last_clean":"2024-09-21T02:58:54.909747+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:58:54.909747+0000","last_undegraded":"2024-09-21T02:58:54.909747+0000","last_fullsized":"2024-09-21T02:58:54.909747+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"77'27","reported_seq":167,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:59:28.772151+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T02:59:28.772151+0000","last_peered":"2024-09-21T02:59:28.772151+0000","last_clean":"2024-09-21T02:59:28.772151+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T02:59:28.772151+0000","last_undegraded":"2024-09-21T02:59:28.772151+0000","last_fullsized":"2024-09-21T02:59:28.772151+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":27,"log_dups_size":0,"ondisk_log_size":27,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":66957,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":123,"num_write":27,"num_write_kb":77,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743238+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:58:54.743238+0000","last_peered":"2024-09-21T02:58:54.743238+0000","last_clean":"2024-09-21T02:58:54.743238+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:58:54.743238+0000","last_undegraded":"2024-09-21T02:58:54.743238+0000","last_fullsized":"2024-09-21T02:58:54.743238+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":26,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741205+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:58:54.741205+0000","last_peered":"2024-09-21T02:58:54.741205+0000","last_clean":"2024-09-21T02:58:54.741205+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:58:54.741205+0000","last_undegraded":"2024-09-21T02:58:54.741205+0000","last_fullsized":"2024-09-21T02:58:54.741205+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741073+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:58:54.741073+0000","last_peered":"2024-09-21T02:58:54.741073+0000","last_clean":"2024-09-21T02:58:54.741073+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:58:54.741073+0000","last_undegraded":"2024-09-21T02:58:54.741073+0000","last_fullsized":"2024-09-21T02:58:54.741073+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738747+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:58:54.738747+0000","last_peered":"2024-09-21T02:58:54.738747+0000","last_clean":"2024-09-21T02:58:54.738747+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:58:54.738747+0000","last_undegraded":"2024-09-21T02:58:54.738747+0000","last_fullsized":"2024-09-21T02:58:54.738747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740670+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:58:54.740670+0000","last_peered":"2024-09-21T02:58:54.740670+0000","last_clean":"2024-09-21T02:58:54.740670+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:58:54.740670+0000","last_undegraded":"2024-09-21T02:58:54.740670+0000","last_fullsized":"2024-09-21T02:58:54.740670+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909289+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:58:54.909289+0000","last_peered":"2024-09-21T02:58:54.909289+0000","last_clean":"2024-09-21T02:58:54.909289+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:58:54.909289+0000","last_undegraded":"2024-09-21T02:58:54.909289+0000","last_fullsized":"2024-09-21T02:58:54.909289+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740143+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:58:54.740143+0000","last_peered":"2024-09-21T02:58:54.740143+0000","last_clean":"2024-09-21T02:58:54.740143+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:58:54.740143+0000","last_undegraded":"2024-09-21T02:58:54.740143+0000","last_fullsized":"2024-09-21T02:58:54.740143+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":31,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.742929+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:58:54.742929+0000","last_peered":"2024-09-21T02:58:54.742929+0000","last_clean":"2024-09-21T02:58:54.742929+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:58:54.742929+0000","last_undegraded":"2024-09-21T02:58:54.742929+0000","last_fullsized":"2024-09-21T02:58:54.742929+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.738601+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:58:54.738601+0000","last_peered":"2024-09-21T02:58:54.738601+0000","last_clean":"2024-09-21T02:58:54.738601+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:58:54.738601+0000","last_undegraded":"2024-09-21T02:58:54.738601+0000","last_fullsized":"2024-09-21T02:58:54.738601+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743275+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:58:54.743275+0000","last_peered":"2024-09-21T02:58:54.743275+0000","last_clean":"2024-09-21T02:58:54.743275+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:58:54.743275+0000","last_undegraded":"2024-09-21T02:58:54.743275+0000","last_fullsized":"2024-09-21T02:58:54.743275+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737834+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:58:54.737834+0000","last_peered":"2024-09-21T02:58:54.737834+0000","last_clean":"2024-09-21T02:58:54.737834+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:58:54.737834+0000","last_undegraded":"2024-09-21T02:58:54.737834+0000","last_fullsized":"2024-09-21T02:58:54.737834+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741546+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:58:54.741546+0000","last_peered":"2024-09-21T02:58:54.741546+0000","last_clean":"2024-09-21T02:58:54.741546+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:58:54.741546+0000","last_undegraded":"2024-09-21T02:58:54.741546+0000","last_fullsized":"2024-09-21T02:58:54.741546+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740954+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:58:54.740954+0000","last_peered":"2024-09-21T02:58:54.740954+0000","last_clean":"2024-09-21T02:58:54.740954+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:58:54.740954+0000","last_undegraded":"2024-09-21T02:58:54.740954+0000","last_fullsized":"2024-09-21T02:58:54.740954+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739462+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:58:54.739462+0000","last_peered":"2024-09-21T02:58:54.739462+0000","last_clean":"2024-09-21T02:58:54.739462+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:58:54.739462+0000","last_undegraded":"2024-09-21T02:58:54.739462+0000","last_fullsized":"2024-09-21T02:58:54.739462+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740785+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:58:54.740785+0000","last_peered":"2024-09-21T02:58:54.740785+0000","last_clean":"2024-09-21T02:58:54.740785+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:58:54.740785+0000","last_undegraded":"2024-09-21T02:58:54.740785+0000","last_fullsized":"2024-09-21T02:58:54.740785+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741748+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:58:54.741748+0000","last_peered":"2024-09-21T02:58:54.741748+0000","last_clean":"2024-09-21T02:58:54.741748+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:58:54.741748+0000","last_undegraded":"2024-09-21T02:58:54.741748+0000","last_fullsized":"2024-09-21T02:58:54.741748+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740858+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:58:54.740858+0000","last_peered":"2024-09-21T02:58:54.740858+0000","last_clean":"2024-09-21T02:58:54.740858+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:58:54.740858+0000","last_undegraded":"2024-09-21T02:58:54.740858+0000","last_fullsized":"2024-09-21T02:58:54.740858+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":93,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740399+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:58:54.740399+0000","last_peered":"2024-09-21T02:58:54.740399+0000","last_clean":"2024-09-21T02:58:54.740399+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:58:54.740399+0000","last_undegraded":"2024-09-21T02:58:54.740399+0000","last_fullsized":"2024-09-21T02:58:54.740399+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909443+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:58:54.909443+0000","last_peered":"2024-09-21T02:58:54.909443+0000","last_clean":"2024-09-21T02:58:54.909443+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:58:54.909443+0000","last_undegraded":"2024-09-21T02:58:54.909443+0000","last_fullsized":"2024-09-21T02:58:54.909443+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736591+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:58:54.736591+0000","last_peered":"2024-09-21T02:58:54.736591+0000","last_clean":"2024-09-21T02:58:54.736591+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:58:54.736591+0000","last_undegraded":"2024-09-21T02:58:54.736591+0000","last_fullsized":"2024-09-21T02:58:54.736591+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740396+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:58:54.740396+0000","last_peered":"2024-09-21T02:58:54.740396+0000","last_clean":"2024-09-21T02:58:54.740396+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:58:54.740396+0000","last_undegraded":"2024-09-21T02:58:54.740396+0000","last_fullsized":"2024-09-21T02:58:54.740396+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740622+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:58:54.740622+0000","last_peered":"2024-09-21T02:58:54.740622+0000","last_clean":"2024-09-21T02:58:54.740622+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:58:54.740622+0000","last_undegraded":"2024-09-21T02:58:54.740622+0000","last_fullsized":"2024-09-21T02:58:54.740622+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740908+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:58:54.740908+0000","last_peered":"2024-09-21T02:58:54.740908+0000","last_clean":"2024-09-21T02:58:54.740908+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:58:54.740908+0000","last_undegraded":"2024-09-21T02:58:54.740908+0000","last_fullsized":"2024-09-21T02:58:54.740908+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909981+0000","last_change":"2024-09-21T02:58:54.909889+0000","last_active":"2024-09-21T02:58:54.909981+0000","last_peered":"2024-09-21T02:58:54.909981+0000","last_clean":"2024-09-21T02:58:54.909981+0000","last_became_active":"2024-09-21T02:58:54.909607+0000","last_became_peered":"2024-09-21T02:58:54.909607+0000","last_unstale":"2024-09-21T02:58:54.909981+0000","last_undegraded":"2024-09-21T02:58:54.909981+0000","last_fullsized":"2024-09-21T02:58:54.909981+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740639+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:58:54.740639+0000","last_peered":"2024-09-21T02:58:54.740639+0000","last_clean":"2024-09-21T02:58:54.740639+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:58:54.740639+0000","last_undegraded":"2024-09-21T02:58:54.740639+0000","last_fullsized":"2024-09-21T02:58:54.740639+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737415+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:58:54.737415+0000","last_peered":"2024-09-21T02:58:54.737415+0000","last_clean":"2024-09-21T02:58:54.737415+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:58:54.737415+0000","last_undegraded":"2024-09-21T02:58:54.737415+0000","last_fullsized":"2024-09-21T02:58:54.737415+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739708+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:58:54.739708+0000","last_peered":"2024-09-21T02:58:54.739708+0000","last_clean":"2024-09-21T02:58:54.739708+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:58:54.739708+0000","last_undegraded":"2024-09-21T02:58:54.739708+0000","last_fullsized":"2024-09-21T02:58:54.739708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"75'1","reported_seq":19,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737407+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:58:54.737407+0000","last_peered":"2024-09-21T02:58:54.737407+0000","last_clean":"2024-09-21T02:58:54.737407+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:58:54.737407+0000","last_undegraded":"2024-09-21T02:58:54.737407+0000","last_fullsized":"2024-09-21T02:58:54.737407+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744125+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:58:54.744125+0000","last_peered":"2024-09-21T02:58:54.744125+0000","last_clean":"2024-09-21T02:58:54.744125+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:58:54.744125+0000","last_undegraded":"2024-09-21T02:58:54.744125+0000","last_fullsized":"2024-09-21T02:58:54.744125+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.741141+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:58:54.741141+0000","last_peered":"2024-09-21T02:58:54.741141+0000","last_clean":"2024-09-21T02:58:54.741141+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:58:54.741141+0000","last_undegraded":"2024-09-21T02:58:54.741141+0000","last_fullsized":"2024-09-21T02:58:54.741141+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"75'9","reported_seq":32,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739911+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:58:54.739911+0000","last_peered":"2024-09-21T02:58:54.739911+0000","last_clean":"2024-09-21T02:58:54.739911+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:58:54.739911+0000","last_undegraded":"2024-09-21T02:58:54.739911+0000","last_fullsized":"2024-09-21T02:58:54.739911+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736819+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:58:54.736819+0000","last_peered":"2024-09-21T02:58:54.736819+0000","last_clean":"2024-09-21T02:58:54.736819+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:58:54.736819+0000","last_undegraded":"2024-09-21T02:58:54.736819+0000","last_fullsized":"2024-09-21T02:58:54.736819+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.910580+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:58:54.910580+0000","last_peered":"2024-09-21T02:58:54.910580+0000","last_clean":"2024-09-21T02:58:54.910580+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:58:54.910580+0000","last_undegraded":"2024-09-21T02:58:54.910580+0000","last_fullsized":"2024-09-21T02:58:54.910580+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743644+0000","last_change":"2024-09-21T02:58:54.743546+0000","last_active":"2024-09-21T02:58:54.743644+0000","last_peered":"2024-09-21T02:58:54.743644+0000","last_clean":"2024-09-21T02:58:54.743644+0000","last_became_active":"2024-09-21T02:58:54.743107+0000","last_became_peered":"2024-09-21T02:58:54.743107+0000","last_unstale":"2024-09-21T02:58:54.743644+0000","last_undegraded":"2024-09-21T02:58:54.743644+0000","last_fullsized":"2024-09-21T02:58:54.743644+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.458721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909028+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:58:54.909028+0000","last_peered":"2024-09-21T02:58:54.909028+0000","last_clean":"2024-09-21T02:58:54.909028+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:58:54.909028+0000","last_undegraded":"2024-09-21T02:58:54.909028+0000","last_fullsized":"2024-09-21T02:58:54.909028+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743822+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:58:54.743822+0000","last_peered":"2024-09-21T02:58:54.743822+0000","last_clean":"2024-09-21T02:58:54.743822+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:58:54.743822+0000","last_undegraded":"2024-09-21T02:58:54.743822+0000","last_fullsized":"2024-09-21T02:58:54.743822+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736508+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:58:54.736508+0000","last_peered":"2024-09-21T02:58:54.736508+0000","last_clean":"2024-09-21T02:58:54.736508+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:58:54.736508+0000","last_undegraded":"2024-09-21T02:58:54.736508+0000","last_fullsized":"2024-09-21T02:58:54.736508+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737700+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:58:54.737700+0000","last_peered":"2024-09-21T02:58:54.737700+0000","last_clean":"2024-09-21T02:58:54.737700+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:58:54.737700+0000","last_undegraded":"2024-09-21T02:58:54.737700+0000","last_fullsized":"2024-09-21T02:58:54.737700+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740708+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:58:54.740708+0000","last_peered":"2024-09-21T02:58:54.740708+0000","last_clean":"2024-09-21T02:58:54.740708+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:58:54.740708+0000","last_undegraded":"2024-09-21T02:58:54.740708+0000","last_fullsized":"2024-09-21T02:58:54.740708+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.736662+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:58:54.736662+0000","last_peered":"2024-09-21T02:58:54.736662+0000","last_clean":"2024-09-21T02:58:54.736662+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:58:54.736662+0000","last_undegraded":"2024-09-21T02:58:54.736662+0000","last_fullsized":"2024-09-21T02:58:54.736662+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":37,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.914626+0000","last_change":"2024-09-21T02:58:54.914527+0000","last_active":"2024-09-21T02:58:54.914626+0000","last_peered":"2024-09-21T02:58:54.914626+0000","last_clean":"2024-09-21T02:58:54.914626+0000","last_became_active":"2024-09-21T02:58:54.910405+0000","last_became_peered":"2024-09-21T02:58:54.910405+0000","last_unstale":"2024-09-21T02:58:54.914626+0000","last_undegraded":"2024-09-21T02:58:54.914626+0000","last_fullsized":"2024-09-21T02:58:54.914626+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:50.419816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740322+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:58:54.740322+0000","last_peered":"2024-09-21T02:58:54.740322+0000","last_clean":"2024-09-21T02:58:54.740322+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:58:54.740322+0000","last_undegraded":"2024-09-21T02:58:54.740322+0000","last_fullsized":"2024-09-21T02:58:54.740322+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739187+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:58:54.739187+0000","last_peered":"2024-09-21T02:58:54.739187+0000","last_clean":"2024-09-21T02:58:54.739187+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:58:54.739187+0000","last_undegraded":"2024-09-21T02:58:54.739187+0000","last_fullsized":"2024-09-21T02:58:54.739187+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":25,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.743489+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:58:54.743489+0000","last_peered":"2024-09-21T02:58:54.743489+0000","last_clean":"2024-09-21T02:58:54.743489+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:58:54.743489+0000","last_undegraded":"2024-09-21T02:58:54.743489+0000","last_fullsized":"2024-09-21T02:58:54.743489+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.909737+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:58:54.909737+0000","last_peered":"2024-09-21T02:58:54.909737+0000","last_clean":"2024-09-21T02:58:54.909737+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:58:54.909737+0000","last_undegraded":"2024-09-21T02:58:54.909737+0000","last_fullsized":"2024-09-21T02:58:54.909737+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":17,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.918634+0000","last_change":"2024-09-21T02:58:54.918104+0000","last_active":"2024-09-21T02:58:54.918634+0000","last_peered":"2024-09-21T02:58:54.918634+0000","last_clean":"2024-09-21T02:58:54.918634+0000","last_became_active":"2024-09-21T02:58:54.911376+0000","last_became_peered":"2024-09-21T02:58:54.911376+0000","last_unstale":"2024-09-21T02:58:54.918634+0000","last_undegraded":"2024-09-21T02:58:54.918634+0000","last_fullsized":"2024-09-21T02:58:54.918634+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:42.992181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":18,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.740360+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:58:54.740360+0000","last_peered":"2024-09-21T02:58:54.740360+0000","last_clean":"2024-09-21T02:58:54.740360+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:58:54.740360+0000","last_undegraded":"2024-09-21T02:58:54.740360+0000","last_fullsized":"2024-09-21T02:58:54.740360+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":24,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.737093+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:58:54.737093+0000","last_peered":"2024-09-21T02:58:54.737093+0000","last_clean":"2024-09-21T02:58:54.737093+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:58:54.737093+0000","last_undegraded":"2024-09-21T02:58:54.737093+0000","last_fullsized":"2024-09-21T02:58:54.737093+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":12,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.744135+0000","last_change":"2024-09-21T02:58:54.743356+0000","last_active":"2024-09-21T02:58:54.744135+0000","last_peered":"2024-09-21T02:58:54.744135+0000","last_clean":"2024-09-21T02:58:54.744135+0000","last_became_active":"2024-09-21T02:58:54.742875+0000","last_became_peered":"2024-09-21T02:58:54.742875+0000","last_unstale":"2024-09-21T02:58:54.744135+0000","last_undegraded":"2024-09-21T02:58:54.744135+0000","last_fullsized":"2024-09-21T02:58:54.744135+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:47.202904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":28,"reported_epoch":77,"state":"active+clean","last_fresh":"2024-09-21T02:58:54.739505+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:58:54.739505+0000","last_peered":"2024-09-21T02:58:54.739505+0000","last_clean":"2024-09-21T02:58:54.739505+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:58:54.739505+0000","last_undegraded":"2024-09-21T02:58:54.739505+0000","last_fullsized":"2024-09-21T02:58:54.739505+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":67755,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":123,"num_read_kb":129,"num_write":83,"num_write_kb":93,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":203265,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":68,"ondisk_log_size":68,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743449,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":816,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":835584,"data_stored":412594,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906975,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":40992,"kb_used_data":1488,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704160,"statfs":{"total":95995035648,"available":95953059840,"internally_reserved":0,"allocated":1523712,"data_stored":1070230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070500,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":839680,"data_stored":413176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234025,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27852,"kb_used_data":824,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717300,"statfs":{"total":95995035648,"available":95966515200,"internally_reserved":0,"allocated":843776,"data_stored":413198,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430253,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36756,"kb_used_data":1404,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708396,"statfs":{"total":95995035648,"available":95957397504,"internally_reserved":0,"allocated":1437696,"data_stored":1003661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593779,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":412711,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757303,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32604,"kb_used_data":1412,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712548,"statfs":{"total":95995035648,"available":95961649152,"internally_reserved":0,"allocated":1445888,"data_stored":1003261,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953532,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36172,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708980,"statfs":{"total":95995035648,"available":95957995520,"internally_reserved":0,"allocated":839680,"data_stored":412616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149761,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27864,"kb_used_data":832,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717288,"statfs":{"total":95995035648,"available":95966502912,"internally_reserved":0,"allocated":851968,"data_stored":412932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378696,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36240,"kb_used_data":896,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708912,"statfs":{"total":95995035648,"available":95957925888,"internally_reserved":0,"allocated":917504,"data_stored":479585,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542219,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27916,"kb_used_data":892,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717236,"statfs":{"total":95995035648,"available":95966449664,"internally_reserved":0,"allocated":913408,"data_stored":479551,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705745,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27848,"kb_used_data":820,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717304,"statfs":{"total":95995035648,"available":95966519296,"internally_reserved":0,"allocated":839680,"data_stored":412684,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":66957,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":66991,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67151,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T02:59:32.565 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-09-21T02:59:32.565 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-09-21T02:59:32.565 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-09-21T02:59:32.565 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph health --format=json 2024-09-21T02:59:32.828 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:32 smithi018 ceph-mon[34587]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:32 smithi018 ceph-mon[34587]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:33.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:32 smithi016 ceph-mon[31253]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:33.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:32 smithi016 ceph-mon[31253]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:32 smithi123 ceph-mon[34645]: pgmap v294: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:32 smithi123 ceph-mon[34645]: from='client.15168 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- 172.21.15.16:0/336465467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61941094a0 msgr2=0x7f61941098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 --2- 172.21.15.16:0/336465467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61941094a0 0x7f61941098b0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f6180009900 tx=0x7f618002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- 172.21.15.16:0/336465467 shutdown_connections 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 --2- 172.21.15.16:0/336465467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f61941094a0 0x7f61941098b0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 --2- 172.21.15.16:0/336465467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6194109df0 0x7f6194110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 --2- 172.21.15.16:0/336465467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194103b60 0x7f6194103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- 172.21.15.16:0/336465467 >> 172.21.15.16:0/336465467 conn(0x7f619407b850 msgr2=0x7f619407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- 172.21.15.16:0/336465467 shutdown_connections 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- 172.21.15.16:0/336465467 wait complete. 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 Processor -- start 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.291+0000 7f619ad68640 1 -- start start 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941094a0 0x7f619419aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 0x7f619419fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6194113b20 con 0x7f6194103b60 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f61941139a0 con 0x7f6194109df0 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f619ad68640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6194113ca0 con 0x7f61941094a0 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:33.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:57794/0 (socket says 172.21.15.16:57794) 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 -- 172.21.15.16:0/1765254569 learned_addr learned my addr 172.21.15.16:0/1765254569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6193fff640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941094a0 0x7f619419aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6199305640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 0x7f619419fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 -- 172.21.15.16:0/1765254569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941094a0 msgr2=0x7f619419aaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941094a0 0x7f619419aaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 -- 172.21.15.16:0/1765254569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 msgr2=0x7f619419fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 0x7f619419fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 -- 172.21.15.16:0/1765254569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f61941a0400 con 0x7f6194103b60 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6199305640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 0x7f619419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.292+0000 7f6198b04640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f618800b4a0 tx=0x7f618800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.293+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188014070 con 0x7f6194103b60 2024-09-21T02:59:33.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.293+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f61941a06f0 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.293+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6188010ac0 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.293+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6188017c30 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.293+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f61941a6f10 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.294+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6194103f70 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.295+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f6188017070 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.296+0000 7f6191ffb640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 0x7f61840b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.296+0000 7f6193fff640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 0x7f61840b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.296+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f61880e6ca0 con 0x7f6194103b60 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.297+0000 7f6193fff640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 0x7f61840b8b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f618002aa00 tx=0x7f6180031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:33.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.299+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f61880ad2f0 con 0x7f6194103b60 2024-09-21T02:59:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.528+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f619419b680 con 0x7f6194103b60 2024-09-21T02:59:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.529+0000 7f6191ffb640 1 -- 172.21.15.16:0/1765254569 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f61880b16d0 con 0x7f6194103b60 2024-09-21T02:59:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:33.530 INFO:teuthology.orchestra.run.smithi016.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-09-21T02:59:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 msgr2=0x7f61840b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:33.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 0x7f61840b8b30 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f618002aa00 tx=0x7f6180031400 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 msgr2=0x7f619419a550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f618800b4a0 tx=0x7f618800b960 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 shutdown_connections 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f61840b6620 0x7f61840b8b30 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6194109df0 0x7f619419fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f61941094a0 0x7f619419aaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 --2- 172.21.15.16:0/1765254569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6194103b60 0x7f619419a550 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 >> 172.21.15.16:0/1765254569 conn(0x7f619407b850 msgr2=0x7f6194107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 shutdown_connections 2024-09-21T02:59:33.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:33.531+0000 7f619ad68640 1 -- 172.21.15.16:0/1765254569 wait complete. 2024-09-21T02:59:34.138 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-09-21T02:59:34.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:34.163 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1765254569' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:59:34.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1765254569' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:59:34.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1765254569' entity='client.admin' cmd={"prefix": "health", "format": "json"} : dispatch 2024-09-21T02:59:34.398 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 -- 172.21.15.16:0/3672763165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc1052a0 msgr2=0x7f58fc10d930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 --2- 172.21.15.16:0/3672763165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc1052a0 0x7f58fc10d930 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f58e4009900 tx=0x7f58e402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 -- 172.21.15.16:0/3672763165 shutdown_connections 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 --2- 172.21.15.16:0/3672763165 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc1052a0 0x7f58fc10d930 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 --2- 172.21.15.16:0/3672763165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc10de70 0x7f58fc110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 --2- 172.21.15.16:0/3672763165 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 0x7f58fc104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 -- 172.21.15.16:0/3672763165 >> 172.21.15.16:0/3672763165 conn(0x7f58fc079760 msgr2=0x7f58fc079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 -- 172.21.15.16:0/3672763165 shutdown_connections 2024-09-21T02:59:34.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.841+0000 7f5900cc2640 1 -- 172.21.15.16:0/3672763165 wait complete. 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 Processor -- start 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 -- start start 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 0x7f58fc19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc1052a0 0x7f58fc19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc113af0 con 0x7f58fc10de70 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc113970 con 0x7f58fc1048c0 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f5900cc2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f58fc113c70 con 0x7f58fc1052a0 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.842+0000 7f58fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52484/0 (socket says 172.21.15.16:52484) 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 -- 172.21.15.16:0/171415931 learned_addr learned my addr 172.21.15.16:0/171415931 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fa59c640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 0x7f58fc19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58f3fff640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc1052a0 0x7f58fc19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 -- 172.21.15.16:0/171415931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc1052a0 msgr2=0x7f58fc19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc1052a0 0x7f58fc19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:34.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 -- 172.21.15.16:0/171415931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 msgr2=0x7f58fc19a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 0x7f58fc19a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 -- 172.21.15.16:0/171415931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f58fc1a04f0 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58fad9d640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f58f400b970 tx=0x7f58f400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f4019070 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f58fc1a07e0 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f58f4014440 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f58f40075d0 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.843+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f58fc1a6f40 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.844+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f58fc06c190 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.846+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f58f401c070 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.847+0000 7f58f37fe640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 0x7f58e80b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.847+0000 7f58fa59c640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 0x7f58e80b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.847+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f58f40e6f30 con 0x7f58fc10de70 2024-09-21T02:59:34.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.847+0000 7f58fa59c640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 0x7f58e80b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58fc06c890 tx=0x7f58ec00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:34.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:34.850+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f58f40b1960 con 0x7f58fc10de70 2024-09-21T02:59:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.080+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f58fc104cd0 con 0x7f58fc10de70 2024-09-21T02:59:35.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.081+0000 7f58f37fe640 1 -- 172.21.15.16:0/171415931 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f58f40b68a0 con 0x7f58fc10de70 2024-09-21T02:59:35.082 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:35.083 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:59:35.083 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 msgr2=0x7f58e80b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 0x7f58e80b8b00 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f58fc06c890 tx=0x7f58ec00b390 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 msgr2=0x7f58fc19fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f58f400b970 tx=0x7f58f400be30 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 shutdown_connections 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f58e80b65f0 0x7f58e80b8b00 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f58fc10de70 0x7f58fc19fdc0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f58fc1052a0 0x7f58fc19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 --2- 172.21.15.16:0/171415931 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f58fc1048c0 0x7f58fc19a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.083+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 >> 172.21.15.16:0/171415931 conn(0x7f58fc079760 msgr2=0x7f58fc100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:35.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.084+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 shutdown_connections 2024-09-21T02:59:35.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:35.084+0000 7f5900cc2640 1 -- 172.21.15.16:0/171415931 wait complete. 2024-09-21T02:59:35.095 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:34 smithi016 ceph-mon[31253]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:34 smithi018 ceph-mon[34587]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:34 smithi123 ceph-mon[34645]: pgmap v295: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:35.719 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:36.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:35 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/171415931' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:36.023 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:35 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/171415931' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:36.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:35 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/171415931' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 -- 172.21.15.16:0/2630180070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12141094c0 msgr2=0x7f12141098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 --2- 172.21.15.16:0/2630180070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12141094c0 0x7f12141098d0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f1208009900 tx=0x7f120802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 -- 172.21.15.16:0/2630180070 shutdown_connections 2024-09-21T02:59:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 --2- 172.21.15.16:0/2630180070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f12141094c0 0x7f12141098d0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 --2- 172.21.15.16:0/2630180070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1214109e10 0x7f1214110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 --2- 172.21.15.16:0/2630180070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214103b80 0x7f1214103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 -- 172.21.15.16:0/2630180070 >> 172.21.15.16:0/2630180070 conn(0x7f12140ff610 msgr2=0x7f1214101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 -- 172.21.15.16:0/2630180070 shutdown_connections 2024-09-21T02:59:36.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.489+0000 7f1219d22640 1 -- 172.21.15.16:0/2630180070 wait complete. 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.490+0000 7f1219d22640 1 Processor -- start 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.490+0000 7f1219d22640 1 -- start start 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.490+0000 7f1219d22640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1219d22640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12141094c0 0x7f121419aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1219d22640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214109e10 0x7f121419fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1219d22640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113ad0 con 0x7f1214103b80 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1219d22640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113950 con 0x7f12141094c0 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1219d22640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1214113c50 con 0x7f1214109e10 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:36.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52508/0 (socket says 172.21.15.16:52508) 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 -- 172.21.15.16:0/4105229608 learned_addr learned my addr 172.21.15.16:0/4105229608 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1212ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12141094c0 0x7f121419aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1213fff640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214109e10 0x7f121419fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 -- 172.21.15.16:0/4105229608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214109e10 msgr2=0x7f121419fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214109e10 0x7f121419fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 -- 172.21.15.16:0/4105229608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12141094c0 msgr2=0x7f121419aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12141094c0 0x7f121419aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 -- 172.21.15.16:0/4105229608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12141a0450 con 0x7f1214103b80 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f12137fe640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f120400d950 tx=0x7f120400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.491+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1204017030 con 0x7f1214103b80 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.492+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12141a0740 con 0x7f1214103b80 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.492+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1204004020 con 0x7f1214103b80 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.492+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12040047a0 con 0x7f1214103b80 2024-09-21T02:59:36.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.492+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12141a6ea0 con 0x7f1214103b80 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.493+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1214103f90 con 0x7f1214103b80 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.494+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f1204009a30 con 0x7f1214103b80 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.495+0000 7f1210ff9640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 0x7f11fc0b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.495+0000 7f1212ffd640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 0x7f11fc0b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.495+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f12040e6fe0 con 0x7f1214103b80 2024-09-21T02:59:36.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.496+0000 7f1212ffd640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 0x7f11fc0b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1208009230 tx=0x7f1208031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:36.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.499+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f12040ad610 con 0x7f1214103b80 2024-09-21T02:59:36.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.727+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f121402e640 con 0x7f1214103b80 2024-09-21T02:59:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.728+0000 7f1210ff9640 1 -- 172.21.15.16:0/4105229608 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f12040b19f0 con 0x7f1214103b80 2024-09-21T02:59:36.730 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:36.730 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:59:36.730 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.730+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 msgr2=0x7f11fc0b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 0x7f11fc0b8b80 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1208009230 tx=0x7f1208031400 comp rx=0 tx=0).stop 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 msgr2=0x7f121419a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f120400d950 tx=0x7f120400de10 comp rx=0 tx=0).stop 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 shutdown_connections 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f11fc0b6670 0x7f11fc0b8b80 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1214109e10 0x7f121419fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f12141094c0 0x7f121419aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 --2- 172.21.15.16:0/4105229608 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1214103b80 0x7f121419a4e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 >> 172.21.15.16:0/4105229608 conn(0x7f12140ff610 msgr2=0x7f1214101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 shutdown_connections 2024-09-21T02:59:36.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:36.731+0000 7f1219d22640 1 -- 172.21.15.16:0/4105229608 wait complete. 2024-09-21T02:59:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:36 smithi016 ceph-mon[31253]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:36 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4105229608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:36 smithi018 ceph-mon[34587]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:36 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4105229608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:36 smithi123 ceph-mon[34645]: pgmap v296: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:36 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4105229608' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:37.433 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:59:37.690 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 -- 172.21.15.16:0/2566266810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1048a0 msgr2=0x7f364c104cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 --2- 172.21.15.16:0/2566266810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1048a0 0x7f364c104cb0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f363c009900 tx=0x7f363c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 -- 172.21.15.16:0/2566266810 shutdown_connections 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 --2- 172.21.15.16:0/2566266810 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c10de50 0x7f364c1102f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 --2- 172.21.15.16:0/2566266810 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1052d0 0x7f364c10d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 --2- 172.21.15.16:0/2566266810 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c1048a0 0x7f364c104cb0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 -- 172.21.15.16:0/2566266810 >> 172.21.15.16:0/2566266810 conn(0x7f364c079730 msgr2=0x7f364c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 -- 172.21.15.16:0/2566266810 shutdown_connections 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.167+0000 7f36537e0640 1 -- 172.21.15.16:0/2566266810 wait complete. 2024-09-21T02:59:38.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 Processor -- start 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 -- start start 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1048a0 0x7f364c19a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c1052d0 0x7f364c19aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f364c113a60 con 0x7f364c10de50 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f364c1138e0 con 0x7f364c1048a0 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f36537e0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f364c113be0 con 0x7f364c1052d0 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f3651d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f3651d7d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52518/0 (socket says 172.21.15.16:52518) 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f3651d7d640 1 -- 172.21.15.16:0/3214378383 learned_addr learned my addr 172.21.15.16:0/3214378383 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.168+0000 7f365157c640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1048a0 0x7f364c19a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3650d7b640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c1052d0 0x7f364c19aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 -- 172.21.15.16:0/3214378383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c1052d0 msgr2=0x7f364c19aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c1052d0 0x7f364c19aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 -- 172.21.15.16:0/3214378383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1048a0 msgr2=0x7f364c19a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1048a0 0x7f364c19a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 -- 172.21.15.16:0/3214378383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f364c1a03e0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f3651d7d640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f364400eaa0 tx=0x7f364400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f364400cdb0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f364c1a06d0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3644005020 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36440116d0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.169+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f364c1a6e30 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.170+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f364c104cb0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.172+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f36440051d0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.173+0000 7f36427fc640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 0x7f36380b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.173+0000 7f365157c640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 0x7f36380b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.173+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f36440e7ba0 con 0x7f364c10de50 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.174+0000 7f365157c640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 0x7f36380b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f363c000c00 tx=0x7f363c031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:38.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.176+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3644016030 con 0x7f364c10de50 2024-09-21T02:59:38.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.365+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f364c19b5a0 con 0x7f364c10de50 2024-09-21T02:59:38.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.366+0000 7f36427fc640 1 -- 172.21.15.16:0/3214378383 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v77) ==== 74+0+23079 (secure 0 0 0) 0x7f364c19b5a0 con 0x7f364c10de50 2024-09-21T02:59:38.368 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:38.368 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":77,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:58:54.732993+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:59:38.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 msgr2=0x7f36380b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:38.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 0x7f36380b8b00 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f363c000c00 tx=0x7f363c031400 comp rx=0 tx=0).stop 2024-09-21T02:59:38.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 msgr2=0x7f364c19fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f364400eaa0 tx=0x7f364400ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 shutdown_connections 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f36380b65f0 0x7f36380b8b00 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f364c10de50 0x7f364c19fcb0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f364c1052d0 0x7f364c19aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 --2- 172.21.15.16:0/3214378383 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f364c1048a0 0x7f364c19a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 >> 172.21.15.16:0/3214378383 conn(0x7f364c079730 msgr2=0x7f364c100920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 shutdown_connections 2024-09-21T02:59:38.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:38.369+0000 7f36537e0640 1 -- 172.21.15.16:0/3214378383 wait complete. 2024-09-21T02:59:39.049 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:39.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:38 smithi016 ceph-mon[31253]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:39.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:38 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3214378383' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:38 smithi018 ceph-mon[34587]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:38 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3214378383' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:38 smithi123 ceph-mon[34645]: pgmap v297: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:38 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3214378383' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:39.306 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 -- 172.21.15.16:0/2276791713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8105340 msgr2=0x7f38e810d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 --2- 172.21.15.16:0/2276791713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8105340 0x7f38e810d890 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009900 tx=0x7f38dc02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 -- 172.21.15.16:0/2276791713 shutdown_connections 2024-09-21T02:59:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 --2- 172.21.15.16:0/2276791713 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e8105340 0x7f38e810d890 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:39.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 --2- 172.21.15.16:0/2276791713 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e810de50 0x7f38e8110340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 --2- 172.21.15.16:0/2276791713 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e81049f0 0x7f38e8104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 -- 172.21.15.16:0/2276791713 >> 172.21.15.16:0/2276791713 conn(0x7f38e80797e0 msgr2=0x7f38e8079bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 -- 172.21.15.16:0/2276791713 shutdown_connections 2024-09-21T02:59:39.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.781+0000 7f38eccc5640 1 -- 172.21.15.16:0/2276791713 wait complete. 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.782+0000 7f38eccc5640 1 Processor -- start 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.782+0000 7f38eccc5640 1 -- start start 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.782+0000 7f38eccc5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.782+0000 7f38eccc5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e8105340 0x7f38e819ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.782+0000 7f38eccc5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e810de50 0x7f38e819fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38eccc5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8113ab0 con 0x7f38e81049f0 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38eccc5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8113930 con 0x7f38e8105340 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38eccc5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f38e8113c30 con 0x7f38e810de50 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52534/0 (socket says 172.21.15.16:52534) 2024-09-21T02:59:39.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 -- 172.21.15.16:0/3399701485 learned_addr learned my addr 172.21.15.16:0/3399701485 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e6d9d640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e810de50 0x7f38e819fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e5d9b640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e8105340 0x7f38e819ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 -- 172.21.15.16:0/3399701485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e810de50 msgr2=0x7f38e819fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e810de50 0x7f38e819fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 -- 172.21.15.16:0/3399701485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e8105340 msgr2=0x7f38e819ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e8105340 0x7f38e819ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 -- 172.21.15.16:0/3399701485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e81a04d0 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38e659c640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f38d800e9c0 tx=0x7f38d800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d800cd80 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d80035f0 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e81a07c0 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d8010690 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.783+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e81a6f20 con 0x7f38e81049f0 2024-09-21T02:59:39.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.784+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e806bf20 con 0x7f38e81049f0 2024-09-21T02:59:39.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.787+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f38d8004000 con 0x7f38e81049f0 2024-09-21T02:59:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.788+0000 7f38d77fe640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 0x7f38d00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.788+0000 7f38e5d9b640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 0x7f38d00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.788+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f38d80e7d10 con 0x7f38e81049f0 2024-09-21T02:59:39.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.788+0000 7f38e5d9b640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 0x7f38d00b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009230 tx=0x7f38dc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:39.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:39.791+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f38d80ae300 con 0x7f38e81049f0 2024-09-21T02:59:40.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.020+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f38e8104e00 con 0x7f38e81049f0 2024-09-21T02:59:40.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.021+0000 7f38d77fe640 1 -- 172.21.15.16:0/3399701485 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f38e8104e00 con 0x7f38e81049f0 2024-09-21T02:59:40.022 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:40.022 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:59:40.023 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:40.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 msgr2=0x7f38d00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 0x7f38d00b8b00 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f38dc009230 tx=0x7f38dc031400 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 msgr2=0x7f38e819a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f38d800e9c0 tx=0x7f38d800ee80 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 shutdown_connections 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f38d00b65f0 0x7f38d00b8b00 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38e810de50 0x7f38e819fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f38e8105340 0x7f38e819ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 --2- 172.21.15.16:0/3399701485 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f38e81049f0 0x7f38e819a560 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 >> 172.21.15.16:0/3399701485 conn(0x7f38e80797e0 msgr2=0x7f38e8100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 shutdown_connections 2024-09-21T02:59:40.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:40.024+0000 7f38eccc5640 1 -- 172.21.15.16:0/3399701485 wait complete. 2024-09-21T02:59:40.634 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:58:28.365002+0000', 'modified': '2024-09-21T02:58:37.482868+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24701}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6837/2180643175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6836', 'nonce': 2180643175}, {'type': 'v1', 'addr': '172.21.15.18:6837', 'nonce': 2180643175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24701': {'gid': 24701, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.18:6839/253530783', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 253530783}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 253530783}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24701, 'qdb_cluster': [24701]} 2024-09-21T02:59:40.634 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:59:40.634 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:59:40.937 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:40 smithi018 ceph-mon[34587]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:40 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3399701485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:41.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:40 smithi016 ceph-mon[31253]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:41.214 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:40 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3399701485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:40 smithi123 ceph-mon[34645]: pgmap v298: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:40 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3399701485' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 -- 172.21.15.16:0/441197550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968105ac0 msgr2=0x7f196810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 --2- 172.21.15.16:0/441197550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968105ac0 0x7f196810f530 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f1960009900 tx=0x7f196002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 -- 172.21.15.16:0/441197550 shutdown_connections 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 --2- 172.21.15.16:0/441197550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968105ac0 0x7f196810f530 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 --2- 172.21.15.16:0/441197550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105170 0x7f1968105580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 --2- 172.21.15.16:0/441197550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968103160 0x7f1968103570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 -- 172.21.15.16:0/441197550 >> 172.21.15.16:0/441197550 conn(0x7f1968077c60 msgr2=0x7f19680780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 -- 172.21.15.16:0/441197550 shutdown_connections 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.422+0000 7f196da75640 1 -- 172.21.15.16:0/441197550 wait complete. 2024-09-21T02:59:41.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 Processor -- start 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 -- start start 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968105170 0x7f1968110550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 0x7f1968110a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1968113ae0 con 0x7f1968103160 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1968113960 con 0x7f1968105170 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f196da75640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1968113c60 con 0x7f1968105ac0 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f19677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f19677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52550/0 (socket says 172.21.15.16:52550) 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f19677fe640 1 -- 172.21.15.16:0/1252286598 learned_addr learned my addr 172.21.15.16:0/1252286598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.423+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968105170 0x7f1968110550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f1967fff640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 0x7f1968110a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 -- 172.21.15.16:0/1252286598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 msgr2=0x7f1968110a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 0x7f1968110a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 -- 172.21.15.16:0/1252286598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968105170 msgr2=0x7f1968110550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968105170 0x7f1968110550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 -- 172.21.15.16:0/1252286598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f19681a86a0 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f1967fff640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 0x7f1968110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f19677fe640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f195800cce0 tx=0x7f195800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f1964ff9640 1 -- 172.21.15.16:0/1252286598 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195800f650 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f1964ff9640 1 -- 172.21.15.16:0/1252286598 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195800fca0 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f19681a8990 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f1964ff9640 1 -- 172.21.15.16:0/1252286598 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1958018b40 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.424+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f19681a8ed0 con 0x7f1968103160 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 Processor -- start 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 -- start start 2024-09-21T02:59:41.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1964ff9640 1 -- 172.21.15.16:0/1252286598 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f195800f7f0 con 0x7f1968103160 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2151f60 0x55bac24f7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac24f8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bac246fc10 con 0x55bac2131280 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bac245f040 con 0x55bac2151f60 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.427+0000 7f1973fe9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bac2462e50 con 0x55bac21290a0 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52558/0 (socket says 172.21.15.16:52558) 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 -- 172.21.15.16:0/1876192056 learned_addr learned my addr 172.21.15.16:0/1876192056 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f19677fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac24f8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1967fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2151f60 0x55bac24f7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 -- 172.21.15.16:0/1876192056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 msgr2=0x55bac24f8440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac24f8440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 -- 172.21.15.16:0/1876192056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2151f60 msgr2=0x55bac24f7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2151f60 0x55bac24f7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 -- 172.21.15.16:0/1876192056 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac21bd540 con 0x55bac2131280 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1964ff9640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 0x7f19500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f195c00aaf0 tx=0x7f195c00f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5960dad565fbe47e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1967fff640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 0x7f19500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f194dffb640 1 -- 172.21.15.16:0/1876192056 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c004020 con 0x55bac2131280 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f1964ff9640 1 -- 172.21.15.16:0/1252286598 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f19580e7ae0 con 0x7f1968103160 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f194dffb640 1 -- 172.21.15.16:0/1876192056 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c0041c0 con 0x55bac2131280 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.428+0000 7f194dffb640 1 -- 172.21.15.16:0/1876192056 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c018760 con 0x55bac2131280 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 -- 172.21.15.16:0/1876192056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 msgr2=0x55bac24f8980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1967fff640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 0x7f19500b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1960031440 tx=0x7f1960009640 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f195c00aaf0 tx=0x7f195c00f3c0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 -- 172.21.15.16:0/1876192056 shutdown_connections 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2131280 0x55bac24f8980 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac24f8440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1876192056 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2151f60 0x55bac24f7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 -- 172.21.15.16:0/1876192056 >> 172.21.15.16:0/1876192056 conn(0x55bac2146350 msgr2=0x55bac24f5df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 -- 172.21.15.16:0/1876192056 shutdown_connections 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.429+0000 7f1973fe9740 1 -- 172.21.15.16:0/1876192056 wait complete. 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 -- start start 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 Processor -- start 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac2503760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2131280 0x55bac2503d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bac22c05f0 con 0x55bac2151f60 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bac23fa010 con 0x55bac2131280 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.430+0000 7f1973fe9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bac25066e0 con 0x55bac21290a0 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1973fe9740 10 client.0 _ll_register_callbacks cb 0x55bac2459340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52564/0 (socket says 172.21.15.16:52564) 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2131280 0x55bac2503d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 -- 172.21.15.16:0/1059653439 learned_addr learned my addr 172.21.15.16:0/1059653439 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f19677fe640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac2503760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 -- 172.21.15.16:0/1059653439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 msgr2=0x55bac2503760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac2503760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 -- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2131280 msgr2=0x55bac2503d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2131280 0x55bac2503d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 -- 172.21.15.16:0/1059653439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac2427a00 con 0x55bac2151f60 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1967fff640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1960033000 tx=0x7f196002a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0e4d39d3a724b9d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f196004c070 con 0x55bac2151f60 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f19600043d0 con 0x55bac2151f60 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.431+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c0041c0 con 0x55bac2151f60 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1960058050 con 0x55bac2151f60 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 10 client.15210 fetch_fsmap learned FSMap version 20 2024-09-21T02:59:41.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bac2512600 con 0x55bac2151f60 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c0041c0 con 0x55bac2151f60 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f196003c640 con 0x55bac2151f60 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 10 client.15210 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 10 client.15210 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 20 client.15210 populate_metadata read hostname 'smithi016' 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac21108e0 0x55bac2516110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 4 client.15210 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bac2471b60 con 0x55bac21108e0 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.432+0000 7f19677fe640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac21108e0 0x55bac2516110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.433+0000 7f19677fe640 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac21108e0 0x55bac2516110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3145870752329482 server_cookie=bb5d0cbec66e3997 in_seq=0 out_seq=0 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.433+0000 7f1946ffd640 10 client.15210 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1946ffd640 1 -- 172.21.15.16:0/1059653439 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bac2471b60 con 0x55bac21108e0 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1946ffd640 10 client.15210 handle_command_reply: tid=0 2024-09-21T02:59:41.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 client.15210 shutdown 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 msgr2=0x55bac250fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ffc7910c-77c3-11ef-baf5-efdc52797490", 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24701, 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 72.820037189000004, 2024-09-21T02:59:41.469 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 20, 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 77, 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 83.673910234000004 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f1960033000 tx=0x7f196002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 shutdown_connections 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac21108e0 0x55bac2516110 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac2151f60 0x55bac250fa30 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1946ffd640 0 client.15210 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:41.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac2131280 0x55bac2503d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1059653439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac21290a0 0x55bac2503760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 >> 172.21.15.16:0/1059653439 conn(0x55bac2146350 msgr2=0x55bac24fb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 shutdown_connections 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 1 -- 172.21.15.16:0/1059653439 wait complete. 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.434+0000 7f1973fe9740 20 client.15210 trim_cache size 0 max 16384 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.452+0000 7f1973fe9740 1 Processor -- start 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.452+0000 7f1973fe9740 1 -- start start 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.452+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 0x55bac2501fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.452+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.452+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac25036d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1973fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bac2514450 con 0x55bac21290a0 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1973fe9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bac2514b20 con 0x55bac21108e0 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1973fe9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bac25151f0 con 0x55bac2131280 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52578/0 (socket says 172.21.15.16:52578) 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 -- 172.21.15.16:0/1314679790 learned_addr learned my addr 172.21.15.16:0/1314679790 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f19677fe640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 0x55bac2501fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 -- 172.21.15.16:0/1314679790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 msgr2=0x55bac25036d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1967fff640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac25036d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac25036d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 -- 172.21.15.16:0/1314679790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 msgr2=0x55bac2501fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 0x55bac2501fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 -- 172.21.15.16:0/1314679790 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bac2504800 con 0x55bac21290a0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f19677fe640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 0x55bac2501fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1967fff640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac25036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f195c003f30 tx=0x7f195c005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43bdbeeb2d8cbd03 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f194d7fa640 1 -- 172.21.15.16:0/1314679790 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c00fe00 con 0x55bac21290a0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f194d7fa640 1 -- 172.21.15.16:0/1314679790 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c004440 con 0x55bac21290a0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.453+0000 7f194d7fa640 1 -- 172.21.15.16:0/1314679790 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f195c003850 con 0x55bac21290a0 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 -- 172.21.15.16:0/1314679790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 msgr2=0x55bac2502520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f195c003f30 tx=0x7f195c005da0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 -- 172.21.15.16:0/1314679790 shutdown_connections 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21290a0 0x55bac2502520 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac25036d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1314679790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21108e0 0x55bac2501fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 -- 172.21.15.16:0/1314679790 >> 172.21.15.16:0/1314679790 conn(0x55bac2146350 msgr2=0x55bac24fe910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 -- 172.21.15.16:0/1314679790 shutdown_connections 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.454+0000 7f1973fe9740 1 -- 172.21.15.16:0/1314679790 wait complete. 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 -- start start 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 Processor -- start 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 0x55bac2519030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac251ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bac2515f90 con 0x55bac21108e0 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bac25158c0 con 0x55bac21290a0 2024-09-21T02:59:41.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bac2516660 con 0x55bac2131280 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1973fe9740 10 client.0 _ll_register_callbacks cb 0x7f1960040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52588/0 (socket says 172.21.15.16:52588) 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 -- 172.21.15.16:0/1562883012 learned_addr learned my addr 172.21.15.16:0/1562883012 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1967fff640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac251ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 0x55bac2519030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 -- 172.21.15.16:0/1562883012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 msgr2=0x55bac251ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac251ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.455+0000 7f19677fe640 1 -- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 msgr2=0x55bac2519030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f19677fe640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 0x55bac2519030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f19677fe640 1 -- 172.21.15.16:0/1562883012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bac251b340 con 0x55bac21108e0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 0x55bac2519030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f19677fe640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19580aaca0 tx=0x7f19580eb160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9286b99e4bf02529 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T02:59:41.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f19580ad380 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f19580a9ca0 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f195c004440 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f19580a91a0 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 10 client.15216 fetch_fsmap learned FSMap version 20 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bac2581cd0 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f195c004440 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f19580a9380 con 0x55bac21108e0 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 10 client.15216 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 10 client.15216 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 20 client.15216 populate_metadata read hostname 'smithi016' 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac2151f60 0x55bac2585620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.456+0000 7f1973fe9740 4 client.15216 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "status"}) -- 0x55bac246d610 con 0x55bac2151f60 2024-09-21T02:59:41.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac2151f60 0x55bac2585620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1946ffd640 10 client.15216 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1966ffd640 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac2151f60 0x55bac2585620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d4a7c8635424915 server_cookie=e36a7ae239ace6bc in_seq=0 out_seq=0 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1946ffd640 1 -- 172.21.15.16:0/1562883012 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bac246d610 con 0x55bac2151f60 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1946ffd640 10 client.15216 handle_command_reply: tid=0 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.457+0000 7f1973fe9740 1 client.15216 shutdown 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 msgr2=0x55bac2518af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19580aaca0 tx=0x7f19580eb160 comp rx=0 tx=0).stop 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 shutdown_connections 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bac2151f60 0x55bac2585620 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bac2131280 0x55bac251ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bac21290a0 0x55bac2519030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1946ffd640 0 client.15216 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 --2- 172.21.15.16:0/1562883012 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bac21108e0 0x55bac2518af0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 >> 172.21.15.16:0/1562883012 conn(0x55bac2146350 msgr2=0x55bac25036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 shutdown_connections 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 1 -- 172.21.15.16:0/1562883012 wait complete. 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f1973fe9740 20 client.15216 trim_cache size 0 max 16384 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 msgr2=0x7f19500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 0x7f19500b8b00 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1960031440 tx=0x7f1960009640 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 msgr2=0x7f1968110010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f195800cce0 tx=0x7f195800a620 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 shutdown_connections 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19500b65f0 0x7f19500b8b00 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1968105ac0 0x7f1968110a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1968105170 0x7f1968110550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 --2- 172.21.15.16:0/1252286598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1968103160 0x7f1968110010 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 >> 172.21.15.16:0/1252286598 conn(0x7f1968077c60 msgr2=0x7f1968106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 shutdown_connections 2024-09-21T02:59:41.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:41.458+0000 7f196da75640 1 -- 172.21.15.16:0/1252286598 wait complete. 2024-09-21T02:59:42.291 INFO:tasks.cephadm:Setup complete, yielding 2024-09-21T02:59:42.291 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:59:42.308 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T02:59:42.308 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph orch status' 2024-09-21T02:59:42.567 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:42.938 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:42 smithi016 ceph-mon[31253]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:43.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2599256752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 msgr2=0x7fe944102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2599256752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944102e70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fe940009900 tx=0x7fe94002a410 comp rx=0 tx=0).stop 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2599256752 shutdown_connections 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2599256752 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe944108cf0 0x7fe94410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2599256752 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9441083a0 0x7fe9441087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2599256752 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944102e70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2599256752 >> 172.21.15.16:0/2599256752 conn(0x7fe944079730 msgr2=0x7fe944079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2599256752 shutdown_connections 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.030+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2599256752 wait complete. 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 Processor -- start 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 -- start start 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 0x7fe944104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:43.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe944108cf0 0x7fe944104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe944112960 con 0x7fe944102a60 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe9441127e0 con 0x7fe9441083a0 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94bb6f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe944112ae0 con 0x7fe944108cf0 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94990b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94990b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52604/0 (socket says 172.21.15.16:52604) 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.031+0000 7fe94990b640 1 -- 172.21.15.16:0/2782622584 learned_addr learned my addr 172.21.15.16:0/2782622584 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94a10c640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe944108cf0 0x7fe944104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94910a640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 0x7fe944104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 -- 172.21.15.16:0/2782622584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe944108cf0 msgr2=0x7fe944104cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe944108cf0 0x7fe944104cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 -- 172.21.15.16:0/2782622584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 msgr2=0x7fe944104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 0x7fe944104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 -- 172.21.15.16:0/2782622584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9441a8700 con 0x7fe944102a60 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94910a640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 0x7fe944104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:43.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94990b640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe940005940 tx=0x7fe9400042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe94003f030 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe940003cb0 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9441a8990 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe940035070 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.032+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9441a8ec0 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.033+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe944102e70 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.036+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fe940042030 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.037+0000 7fe93affd640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 0x7fe9300b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.037+0000 7fe94910a640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 0x7fe9300b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.037+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fe9401081b0 con 0x7fe944102a60 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.037+0000 7fe94910a640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 0x7fe9300b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe944100b70 tx=0x7fe934005ec0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:43.050 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.040+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe9400ce7a0 con 0x7fe944102a60 2024-09-21T02:59:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:42 smithi018 ceph-mon[34587]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:42 smithi123 ceph-mon[34645]: pgmap v299: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:43.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.246+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fe944106320 con 0x7fe9300b6670 2024-09-21T02:59:43.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.247+0000 7fe93affd640 1 -- 172.21.15.16:0/2782622584 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fe944106320 con 0x7fe9300b6670 2024-09-21T02:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:Backend: cephadm 2024-09-21T02:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:Available: Yes 2024-09-21T02:59:43.249 INFO:teuthology.orchestra.run.smithi016.stdout:Paused: No 2024-09-21T02:59:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 msgr2=0x7fe9300b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 0x7fe9300b8b80 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fe944100b70 tx=0x7fe934005ec0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 msgr2=0x7fe944104250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fe940005940 tx=0x7fe9400042f0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 shutdown_connections 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe9300b6670 0x7fe9300b8b80 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe944108cf0 0x7fe944104cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe9441083a0 0x7fe944104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 --2- 172.21.15.16:0/2782622584 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe944102a60 0x7fe944104250 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 >> 172.21.15.16:0/2782622584 conn(0x7fe944079730 msgr2=0x7fe944101ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 shutdown_connections 2024-09-21T02:59:43.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:43.250+0000 7fe94bb6f640 1 -- 172.21.15.16:0/2782622584 wait complete. 2024-09-21T02:59:43.929 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ps' 2024-09-21T02:59:44.190 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 -- 172.21.15.16:0/184389284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 msgr2=0x7f2d98103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/184389284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d98103fd0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f2d88009ca0 tx=0x7f2d8802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 -- 172.21.15.16:0/184389284 shutdown_connections 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/184389284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d98109e00 0x7f2d98110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/184389284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d981094b0 0x7f2d981098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/184389284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d98103fd0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 -- 172.21.15.16:0/184389284 >> 172.21.15.16:0/184389284 conn(0x7f2d9807b910 msgr2=0x7f2d9807bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 -- 172.21.15.16:0/184389284 shutdown_connections 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.662+0000 7f2d9f68b640 1 -- 172.21.15.16:0/184389284 wait complete. 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 Processor -- start 2024-09-21T02:59:44.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 -- start start 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d981094b0 0x7f2d9819aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d98109e00 0x7f2d9819fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98113ac0 con 0x7f2d98103bc0 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98113940 con 0x7f2d981094b0 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.663+0000 7f2d9f68b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2d98113c40 con 0x7f2d98109e00 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33956/0 (socket says 172.21.15.16:33956) 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 -- 172.21.15.16:0/1219995938 learned_addr learned my addr 172.21.15.16:0/1219995938 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9cc26640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d981094b0 0x7f2d9819aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9dc28640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d98109e00 0x7f2d9819fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 -- 172.21.15.16:0/1219995938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d98109e00 msgr2=0x7f2d9819fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d98109e00 0x7f2d9819fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.681 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 -- 172.21.15.16:0/1219995938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d981094b0 msgr2=0x7f2d9819aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d981094b0 0x7f2d9819aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 -- 172.21.15.16:0/1219995938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d981a03a0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9d427640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802ade0 tx=0x7f2d88003970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d88003d60 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d880044f0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d981a0630 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d8802fd10 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.664+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d981a6eb0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.665+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d98104df0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.668+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f2d880046a0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.669+0000 7f2d967fc640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 0x7f2d800b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.669+0000 7f2d9cc26640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 0x7f2d800b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.669+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f2d88108bc0 con 0x7f2d98103bc0 2024-09-21T02:59:44.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.669+0000 7f2d9cc26640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 0x7f2d800b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d98105bf0 tx=0x7f2d8c00a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:44.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.672+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2d880cf1b0 con 0x7f2d98103bc0 2024-09-21T02:59:44.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.871+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7f2d98063800 con 0x7f2d800b66a0 2024-09-21T02:59:44.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.884+0000 7f2d967fc640 1 -- 172.21.15.16:0/1219995938 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4185 (secure 0 0 0) 0x7f2d98063800 con 0x7f2d800b66a0 2024-09-21T02:59:44.885 INFO:teuthology.orchestra.run.smithi016.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.a smithi016 running (90s) 13s ago 89s 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 119aca2f45a1 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.b smithi018 running (87s) 20s ago 87s 93.5M - 19.3.0-5120-g18432218 ddce307e1f32 216605ce0384 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.c smithi123 running (101s) 20s ago 100s 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 5dd1953510ee 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.d smithi016 running (109s) 13s ago 108s 84.4M - 19.3.0-5120-g18432218 ddce307e1f32 ce70b281ac98 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.e smithi018 running (106s) 20s ago 105s 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 e339fbc1d048 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.f smithi123 running (103s) 20s ago 103s 81.7M - 19.3.0-5120-g18432218 ddce307e1f32 094d54b0ac96 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.g smithi016 running (93s) 13s ago 92s 84.3M - 19.3.0-5120-g18432218 ddce307e1f32 6bb2634aef8d 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.h smithi018 running (98s) 20s ago 98s 86.4M - 19.3.0-5120-g18432218 ddce307e1f32 388109d1357b 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.i smithi123 running (96s) 20s ago 95s 84.3M - 19.3.0-5120-g18432218 ddce307e1f32 24ca7dfb6320 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.j smithi016 running (112s) 13s ago 111s 81.9M - 19.3.0-5120-g18432218 ddce307e1f32 fcd4d697c495 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.k smithi018 running (115s) 20s ago 115s 81.8M - 19.3.0-5120-g18432218 ddce307e1f32 5c241aec2a5b 2024-09-21T02:59:44.886 INFO:teuthology.orchestra.run.smithi016.stdout:mds.l smithi123 running (118s) 20s ago 117s 84.5M - 19.3.0-5120-g18432218 ddce307e1f32 3aead19d935a 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.x smithi016 *:9283,8765 running (9m) 13s ago 9m 585M - 19.3.0-5120-g18432218 ddce307e1f32 136364bf88ff 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.y smithi018 *:8443,8765 running (7m) 20s ago 7m 525M - 19.3.0-5120-g18432218 ddce307e1f32 fce0394e3336 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mgr.z smithi123 *:8443,8765 running (7m) 20s ago 7m 523M - 19.3.0-5120-g18432218 ddce307e1f32 20b935b1def7 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mon.a smithi016 running (9m) 13s ago 9m 120M 2048M 19.3.0-5120-g18432218 ddce307e1f32 9f2db1d0d395 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mon.b smithi018 running (8m) 20s ago 8m 109M 2048M 19.3.0-5120-g18432218 ddce307e1f32 da9c74c5bb83 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:mon.c smithi123 running (8m) 20s ago 8m 110M 2048M 19.3.0-5120-g18432218 ddce307e1f32 4bfb0da36942 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 smithi016 running (6m) 13s ago 6m 116M 4273M 19.3.0-5120-g18432218 ddce307e1f32 0a0eacf41db0 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 smithi018 running (6m) 20s ago 6m 115M 4273M 19.3.0-5120-g18432218 ddce307e1f32 594cda2bfe8a 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 smithi123 running (6m) 20s ago 6m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 d60c740190bb 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 smithi016 running (5m) 13s ago 5m 117M 4273M 19.3.0-5120-g18432218 ddce307e1f32 5868d3195157 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 smithi018 running (5m) 20s ago 5m 116M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3a7d10f0f3f9 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 smithi123 running (4m) 20s ago 4m 123M 4272M 19.3.0-5120-g18432218 ddce307e1f32 4196fe09e6cf 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 smithi016 running (4m) 13s ago 4m 117M 4273M 19.3.0-5120-g18432218 ddce307e1f32 4e6915b363b3 2024-09-21T02:59:44.887 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 smithi018 running (3m) 20s ago 3m 122M 4273M 19.3.0-5120-g18432218 ddce307e1f32 166b3b23c32b 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 smithi123 running (3m) 20s ago 3m 113M 4272M 19.3.0-5120-g18432218 ddce307e1f32 87be1a1ec589 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 smithi016 running (3m) 13s ago 3m 112M 4273M 19.3.0-5120-g18432218 ddce307e1f32 3695efade6e0 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 smithi018 running (2m) 20s ago 2m 122M 4273M 19.3.0-5120-g18432218 ddce307e1f32 81c12857242e 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 smithi123 running (2m) 20s ago 2m 110M 4272M 19.3.0-5120-g18432218 ddce307e1f32 6245cab6354c 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 msgr2=0x7f2d800b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 0x7f2d800b8bb0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f2d98105bf0 tx=0x7f2d8c00a2d0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 msgr2=0x7f2d9819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f2d8802ade0 tx=0x7f2d88003970 comp rx=0 tx=0).stop 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 shutdown_connections 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2d800b66a0 0x7f2d800b8bb0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2d98109e00 0x7f2d9819fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2d981094b0 0x7f2d9819aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 --2- 172.21.15.16:0/1219995938 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2d98103bc0 0x7f2d9819a4f0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.886+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 >> 172.21.15.16:0/1219995938 conn(0x7f2d9807b910 msgr2=0x7f2d981075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.887+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 shutdown_connections 2024-09-21T02:59:44.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:44.887+0000 7f2d9f68b640 1 -- 172.21.15.16:0/1219995938 wait complete. 2024-09-21T02:59:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:44 smithi018 ceph-mon[34587]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:44 smithi018 ceph-mon[34587]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:44 smithi123 ceph-mon[34645]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:44 smithi123 ceph-mon[34645]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:44 smithi016 ceph-mon[31253]: pgmap v300: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:45.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:44 smithi016 ceph-mon[31253]: from='client.15222 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:45.659 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph orch ls' 2024-09-21T02:59:45.917 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:46.171 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:45 smithi016 ceph-mon[31253]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:46.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:45 smithi018 ceph-mon[34587]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:46.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:45 smithi123 ceph-mon[34645]: from='client.15228 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 -- 172.21.15.16:0/1613530907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d041094a0 msgr2=0x7f7d041098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 --2- 172.21.15.16:0/1613530907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d041094a0 0x7f7d041098b0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009900 tx=0x7f7cec02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 -- 172.21.15.16:0/1613530907 shutdown_connections 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 --2- 172.21.15.16:0/1613530907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d041094a0 0x7f7d041098b0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 --2- 172.21.15.16:0/1613530907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04109df0 0x7f7d04110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 --2- 172.21.15.16:0/1613530907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d04103b60 0x7f7d04103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 -- 172.21.15.16:0/1613530907 >> 172.21.15.16:0/1613530907 conn(0x7f7d0407b850 msgr2=0x7f7d0407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 -- 172.21.15.16:0/1613530907 shutdown_connections 2024-09-21T02:59:46.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.387+0000 7f7d09af6640 1 -- 172.21.15.16:0/1613530907 wait complete. 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.388+0000 7f7d09af6640 1 Processor -- start 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.388+0000 7f7d09af6640 1 -- start start 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.388+0000 7f7d09af6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04103b60 0x7f7d04075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.388+0000 7f7d09af6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d041094a0 0x7f7d04075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.388+0000 7f7d09af6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d09af6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04113ab0 con 0x7f7d04109df0 2024-09-21T02:59:46.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d09af6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04113930 con 0x7f7d04103b60 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d09af6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7d04113c30 con 0x7f7d041094a0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33968/0 (socket says 172.21.15.16:33968) 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 -- 172.21.15.16:0/2838430635 learned_addr learned my addr 172.21.15.16:0/2838430635 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7cfaffd640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d041094a0 0x7f7d04075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d037fe640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04103b60 0x7f7d04075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 -- 172.21.15.16:0/2838430635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d041094a0 msgr2=0x7f7d04075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d041094a0 0x7f7d04075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 -- 172.21.15.16:0/2838430635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04103b60 msgr2=0x7f7d04075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04103b60 0x7f7d04075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 -- 172.21.15.16:0/2838430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d041a8840 con 0x7f7d04109df0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d03fff640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00eaa0 tx=0x7f7cfc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc00cdb0 con 0x7f7d04109df0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d041a8b30 con 0x7f7d04109df0 2024-09-21T02:59:46.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cfc005020 con 0x7f7d04109df0 2024-09-21T02:59:46.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.389+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cfc0116d0 con 0x7f7d04109df0 2024-09-21T02:59:46.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.390+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d041a8ff0 con 0x7f7d04109df0 2024-09-21T02:59:46.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.391+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d04103f70 con 0x7f7d04109df0 2024-09-21T02:59:46.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.393+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7cfc0051d0 con 0x7f7d04109df0 2024-09-21T02:59:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.394+0000 7f7d017fa640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 0x7f7cf00b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.394+0000 7f7d037fe640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 0x7f7cf00b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:46.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.394+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f7cfc0e7b00 con 0x7f7d04109df0 2024-09-21T02:59:46.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.394+0000 7f7d037fe640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 0x7f7cf00b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7d04077160 tx=0x7f7cf400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:46.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.397+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7cfc016030 con 0x7f7d04109df0 2024-09-21T02:59:46.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.596+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7f7d04063800 con 0x7f7cf00b65f0 2024-09-21T02:59:46.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.602+0000 7f7d017fa640 1 -- 172.21.15.16:0/2838430635 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7f7d04063800 con 0x7f7cf00b65f0 2024-09-21T02:59:46.604 INFO:teuthology.orchestra.run.smithi016.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-09-21T02:59:46.604 INFO:teuthology.orchestra.run.smithi016.stdout:mds.all 12/12 22s ago 2m smithi016=a;smithi016=d;smithi016=g;smithi016=j;smithi018=b;smithi018=e;smithi018=h;smithi018=k;smithi123=c;smithi123=f;smithi123=i;smithi123=l;count:12 2024-09-21T02:59:46.604 INFO:teuthology.orchestra.run.smithi016.stdout:mgr 3/3 22s ago 7m smithi016=x;smithi018=y;smithi123=z;count:3 2024-09-21T02:59:46.604 INFO:teuthology.orchestra.run.smithi016.stdout:mon 3/3 22s ago 8m smithi016:172.21.15.16=a;smithi018:172.21.15.18=b;smithi123:172.21.15.123=c;count:3 2024-09-21T02:59:46.604 INFO:teuthology.orchestra.run.smithi016.stdout:osd 12 22s ago - 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 msgr2=0x7f7cf00b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 0x7f7cf00b8b00 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f7d04077160 tx=0x7f7cf400b390 comp rx=0 tx=0).stop 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 msgr2=0x7f7d04076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f7cfc00eaa0 tx=0x7f7cfc00ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 shutdown_connections 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7cf00b65f0 0x7f7cf00b8b00 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7d04109df0 0x7f7d04076190 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7d041094a0 0x7f7d04075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.604+0000 7f7d09af6640 1 --2- 172.21.15.16:0/2838430635 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7d04103b60 0x7f7d04075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.605+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 >> 172.21.15.16:0/2838430635 conn(0x7f7d0407b850 msgr2=0x7f7d04105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.605+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 shutdown_connections 2024-09-21T02:59:46.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:46.605+0000 7f7d09af6640 1 -- 172.21.15.16:0/2838430635 wait complete. 2024-09-21T02:59:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:46 smithi018 ceph-mon[34587]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:46 smithi123 ceph-mon[34645]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:47.242 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:46 smithi016 ceph-mon[31253]: pgmap v301: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:47.272 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph orch host ls' 2024-09-21T02:59:47.532 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:48.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 -- 172.21.15.16:0/84938538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 msgr2=0x7f7b30110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:48.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 --2- 172.21.15.16:0/84938538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b30110620 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f7b28009ca0 tx=0x7f7b2802a8e0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 -- 172.21.15.16:0/84938538 shutdown_connections 2024-09-21T02:59:48.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 --2- 172.21.15.16:0/84938538 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b30110620 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 --2- 172.21.15.16:0/84938538 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 0x7f7b301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 --2- 172.21.15.16:0/84938538 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 0x7f7b30103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 -- 172.21.15.16:0/84938538 >> 172.21.15.16:0/84938538 conn(0x7f7b3007b850 msgr2=0x7f7b3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 -- 172.21.15.16:0/84938538 shutdown_connections 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.000+0000 7f7b34873640 1 -- 172.21.15.16:0/84938538 wait complete. 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 Processor -- start 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 -- start start 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 0x7f7b3019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 0x7f7b3019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113af0 con 0x7f7b30109df0 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113970 con 0x7f7b30103b60 2024-09-21T02:59:48.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b34873640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7b30113c70 con 0x7f7b301094a0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b2ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33994/0 (socket says 172.21.15.16:33994) 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.001+0000 7f7b2ed9d640 1 -- 172.21.15.16:0/1276117386 learned_addr learned my addr 172.21.15.16:0/1276117386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2dd9b640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 0x7f7b3019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2e59c640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 0x7f7b3019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 -- 172.21.15.16:0/1276117386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 msgr2=0x7f7b3019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 0x7f7b3019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 -- 172.21.15.16:0/1276117386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 msgr2=0x7f7b3019a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 0x7f7b3019a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 -- 172.21.15.16:0/1276117386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b301a0390 con 0x7f7b30109df0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b2ed9d640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005cc0 tx=0x7f7b28003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:48.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28004040 con 0x7f7b30109df0 2024-09-21T02:59:48.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b28004690 con 0x7f7b30109df0 2024-09-21T02:59:48.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7b28030d90 con 0x7f7b30109df0 2024-09-21T02:59:48.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b301a0620 con 0x7f7b30109df0 2024-09-21T02:59:48.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.002+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b301a0b10 con 0x7f7b30109df0 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.003+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b30103f70 con 0x7f7b30109df0 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.005+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f7b280041e0 con 0x7f7b30109df0 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.006+0000 7f7b1f7fe640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 0x7f7b180b8b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.006+0000 7f7b2e59c640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 0x7f7b180b8b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.006+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f7b28108ab0 con 0x7f7b30109df0 2024-09-21T02:59:48.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.006+0000 7f7b2e59c640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 0x7f7b180b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000abe0 tx=0x7f7b2000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:48.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.009+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7b280cf100 con 0x7f7b30109df0 2024-09-21T02:59:48.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:47 smithi018 ceph-mon[34587]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:48.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.201+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f7b30063800 con 0x7f7b180b6620 2024-09-21T02:59:48.203 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:47 smithi016 ceph-mon[31253]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.202+0000 7f7b1f7fe640 1 -- 172.21.15.16:0/1276117386 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f7b30063800 con 0x7f7b180b6620 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stdout:HOST ADDR LABELS STATUS 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 172.21.15.16 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stdout:smithi018 172.21.15.18 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stdout:smithi123 172.21.15.123 2024-09-21T02:59:48.204 INFO:teuthology.orchestra.run.smithi016.stdout:3 hosts in cluster 2024-09-21T02:59:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.204+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 msgr2=0x7f7b180b8b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.204+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 0x7f7b180b8b30 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f7b2000abe0 tx=0x7f7b2000a2d0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 msgr2=0x7f7b3019fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f7b28005cc0 tx=0x7f7b28003ba0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 shutdown_connections 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7b180b6620 0x7f7b180b8b30 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7b30109df0 0x7f7b3019fd20 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7b301094a0 0x7f7b3019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 --2- 172.21.15.16:0/1276117386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7b30103b60 0x7f7b3019a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 >> 172.21.15.16:0/1276117386 conn(0x7f7b3007b850 msgr2=0x7f7b30107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 shutdown_connections 2024-09-21T02:59:48.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:48.205+0000 7f7b34873640 1 -- 172.21.15.16:0/1276117386 wait complete. 2024-09-21T02:59:48.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:47 smithi123 ceph-mon[34645]: from='client.15234 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:48.855 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph orch device ls' 2024-09-21T02:59:48.961 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:48 smithi016 ceph-mon[31253]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:48.961 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:48 smithi016 ceph-mon[31253]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:49.127 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:48 smithi018 ceph-mon[34587]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:48 smithi018 ceph-mon[34587]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:48 smithi123 ceph-mon[34645]: pgmap v302: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:48 smithi123 ceph-mon[34645]: from='client.15240 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:49.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 -- 172.21.15.16:0/1272920911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 msgr2=0x7f46181024b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 --2- 172.21.15.16:0/1272920911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f46181024b0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f4608009900 tx=0x7f460802a410 comp rx=0 tx=0).stop 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 -- 172.21.15.16:0/1272920911 shutdown_connections 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 --2- 172.21.15.16:0/1272920911 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4618106c40 0x7f46181106b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 --2- 172.21.15.16:0/1272920911 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f46181062f0 0x7f4618106700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 --2- 172.21.15.16:0/1272920911 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f46181024b0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 -- 172.21.15.16:0/1272920911 >> 172.21.15.16:0/1272920911 conn(0x7f4618079f90 msgr2=0x7f461807a3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 -- 172.21.15.16:0/1272920911 shutdown_connections 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 -- 172.21.15.16:0/1272920911 wait complete. 2024-09-21T02:59:49.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.596+0000 7f461e0f8640 1 Processor -- start 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 -- start start 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46181062f0 0x7f4618103580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4618106c40 0x7f4618103ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113af0 con 0x7f46181020a0 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113970 con 0x7f46181062f0 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f461e0f8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4618113c70 con 0x7f4618106c40 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34012/0 (socket says 172.21.15.16:34012) 2024-09-21T02:59:49.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 -- 172.21.15.16:0/2400476649 learned_addr learned my addr 172.21.15.16:0/2400476649 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f4616ffd640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46181062f0 0x7f4618103580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f4617fff640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4618106c40 0x7f4618103ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 -- 172.21.15.16:0/2400476649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4618106c40 msgr2=0x7f4618103ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4618106c40 0x7f4618103ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.597+0000 7f46177fe640 1 -- 172.21.15.16:0/2400476649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46181062f0 msgr2=0x7f4618103580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f46177fe640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46181062f0 0x7f4618103580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f46177fe640 1 -- 172.21.15.16:0/2400476649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46181a8860 con 0x7f46181020a0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f46177fe640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4608005940 tx=0x7f46080040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f460803f030 con 0x7f46181020a0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46181a8af0 con 0x7f46181020a0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46080044c0 con 0x7f46181020a0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4608035070 con 0x7f46181020a0 2024-09-21T02:59:49.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.598+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46181a9020 con 0x7f46181020a0 2024-09-21T02:59:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.599+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46181024b0 con 0x7f46181020a0 2024-09-21T02:59:49.602 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.601+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f4608042030 con 0x7f46181020a0 2024-09-21T02:59:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.601+0000 7f4614ff9640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 0x7f46000b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.602+0000 7f4616ffd640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 0x7f46000b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.602+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f4608107b30 con 0x7f46181020a0 2024-09-21T02:59:49.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.602+0000 7f4616ffd640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 0x7f46000b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f461806b180 tx=0x7f460c00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:49.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.606+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f46080ce180 con 0x7f46181020a0 2024-09-21T02:59:49.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.804+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f4618063800 con 0x7f46000b65f0 2024-09-21T02:59:49.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.805+0000 7f4614ff9640 1 -- 172.21.15.16:0/2400476649 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f4618063800 con 0x7f46000b65f0 2024-09-21T02:59:49.807 INFO:teuthology.orchestra.run.smithi016.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-09-21T02:59:49.807 INFO:teuthology.orchestra.run.smithi016.stdout:smithi016 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB7513002C480DGN 447G No 3m ago Has a FileSystem, LVM detected 2024-09-21T02:59:49.807 INFO:teuthology.orchestra.run.smithi016.stdout:smithi018 /dev/nvme0n1 ssd INTEL_SSDPED1D480GA_PHMB75120091480DGN 447G No 2m ago Has a FileSystem, LVM detected 2024-09-21T02:59:49.807 INFO:teuthology.orchestra.run.smithi016.stdout:smithi123 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400YK400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 msgr2=0x7f46000b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 0x7f46000b8b00 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f461806b180 tx=0x7f460c00b410 comp rx=0 tx=0).stop 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 msgr2=0x7f4618103040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f4608005940 tx=0x7f46080040c0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 shutdown_connections 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f46000b65f0 0x7f46000b8b00 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4618106c40 0x7f4618103ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f46181062f0 0x7f4618103580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 --2- 172.21.15.16:0/2400476649 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f46181020a0 0x7f4618103040 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.807+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 >> 172.21.15.16:0/2400476649 conn(0x7f4618079f90 msgr2=0x7f46181075a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.808+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 shutdown_connections 2024-09-21T02:59:49.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:49.808+0000 7f461e0f8640 1 -- 172.21.15.16:0/2400476649 wait complete. 2024-09-21T02:59:50.190 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:49 smithi016 ceph-mon[31253]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:50.190 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:49 smithi016 ceph-mon[31253]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:49 smithi123 ceph-mon[34645]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:50.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:49 smithi123 ceph-mon[34645]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:50.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:49 smithi018 ceph-mon[34587]: pgmap v303: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:50.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:49 smithi018 ceph-mon[34587]: from='client.15246 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T02:59:50.494 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-09-21T02:59:50.503 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T02:59:50.503 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph fs dump' 2024-09-21T02:59:50.762 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 -- 172.21.15.16:0/3050977987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d41094a0 msgr2=0x7f63d41098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 --2- 172.21.15.16:0/3050977987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d41094a0 0x7f63d41098b0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f63c4009900 tx=0x7f63c402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 -- 172.21.15.16:0/3050977987 shutdown_connections 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 --2- 172.21.15.16:0/3050977987 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d41094a0 0x7f63d41098b0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 --2- 172.21.15.16:0/3050977987 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 0x7f63d4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 --2- 172.21.15.16:0/3050977987 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d4103b60 0x7f63d4103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 -- 172.21.15.16:0/3050977987 >> 172.21.15.16:0/3050977987 conn(0x7f63d407b850 msgr2=0x7f63d407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 -- 172.21.15.16:0/3050977987 shutdown_connections 2024-09-21T02:59:51.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.220+0000 7f63dc45d640 1 -- 172.21.15.16:0/3050977987 wait complete. 2024-09-21T02:59:51.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 Processor -- start 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 -- start start 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d41094a0 0x7f63d419aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 0x7f63d419fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4113b00 con 0x7f63d4103b60 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4113980 con 0x7f63d41094a0 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63dc45d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63d4113c80 con 0x7f63d4109df0 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.221+0000 7f63da1f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34042/0 (socket says 172.21.15.16:34042) 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 -- 172.21.15.16:0/1690936670 learned_addr learned my addr 172.21.15.16:0/1690936670 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63d99f8640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d41094a0 0x7f63d419aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da9fa640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 0x7f63d419fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 -- 172.21.15.16:0/1690936670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 msgr2=0x7f63d419fcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 0x7f63d419fcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 -- 172.21.15.16:0/1690936670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d41094a0 msgr2=0x7f63d419aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d41094a0 0x7f63d419aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 -- 172.21.15.16:0/1690936670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f63d41a03e0 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63da1f9640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f63d000bd30 tx=0x7f63d0009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004020 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63d00054d0 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63d41a06d0 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63d0004bc0 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.222+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63d40755f0 con 0x7f63d4103b60 2024-09-21T02:59:51.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.223+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63d4103f70 con 0x7f63d4103b60 2024-09-21T02:59:51.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.226+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f63d0004d60 con 0x7f63d4103b60 2024-09-21T02:59:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.227+0000 7f63cb7fe640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 0x7f63c00b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.227+0000 7f63d99f8640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 0x7f63c00b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.227+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7f63d00eb5b0 con 0x7f63d4103b60 2024-09-21T02:59:51.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.227+0000 7f63d99f8640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 0x7f63c00b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63c402aa00 tx=0x7f63c4005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:51.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.230+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f63d00b5fe0 con 0x7f63d4103b60 2024-09-21T02:59:51.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.460+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f63d402e640 con 0x7f63d4103b60 2024-09-21T02:59:51.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.460+0000 7f63cb7fe640 1 -- 172.21.15.16:0/1690936670 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 20 v20) ==== 76+0+3046 (secure 0 0 0) 0x7f63d00baf20 con 0x7f63d4103b60 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:e20 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:btime 2024-09-21T02:58:37:670718+0000 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:legacy client fscid: 1 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:Filesystem 'cephfs' (1) 2024-09-21T02:59:51.462 INFO:teuthology.orchestra.run.smithi016.stdout:fs_name cephfs 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 20 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-21T02:58:28.365002+0000 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-21T02:58:37.482868+0000 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:tableserver 0 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:root 0 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:session_timeout 300 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:session_autoclose 300 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:max_file_size 1099511627776 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:max_xattr_size 65536 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:required_client_features {} 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure 0 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:last_failure_osd_epoch 0 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:max_mds 1 2024-09-21T02:59:51.463 INFO:teuthology.orchestra.run.smithi016.stdout:in 0 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:up {0=24701} 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:failed 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:damaged 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:stopped 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:data_pools [3] 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:metadata_pool 2 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:inline_data disabled 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:balancer 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:bal_rank_mask -1 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:standby_count_wanted 0 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:qdb_cluster leader: 24701 members: 24701 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.b{0:24701} state up:active seq 4 addr [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.h{0:24674} state up:standby-replay seq 1 addr [v2:172.21.15.18:6836/2180643175,v1:172.21.15.18:6837/2180643175] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:51.464 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:Standby daemons: 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.j{-1:14772} state up:standby seq 1 addr [v2:172.21.15.16:6834/3714505775,v1:172.21.15.16:6835/3714505775] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.d{-1:14784} state up:standby seq 1 addr [v2:172.21.15.16:6836/2928144675,v1:172.21.15.16:6837/2928144675] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.e{-1:14796} state up:standby seq 1 addr [v2:172.21.15.18:6834/3014746634,v1:172.21.15.18:6835/3014746634] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.g{-1:14844} state up:standby seq 1 addr [v2:172.21.15.16:6838/2173086680,v1:172.21.15.16:6839/2173086680] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.a{-1:14850} state up:standby seq 1 addr [v2:172.21.15.16:6840/3872585849,v1:172.21.15.16:6841/3872585849] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.l{-1:24604} state up:standby seq 1 addr [v2:172.21.15.123:6832/3810750401,v1:172.21.15.123:6833/3810750401] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.k{-1:24617} state up:standby seq 1 addr [v2:172.21.15.18:6832/3267677843,v1:172.21.15.18:6833/3267677843] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.f{-1:24643} state up:standby seq 1 addr [v2:172.21.15.123:6834/4189028695,v1:172.21.15.123:6835/4189028695] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.c{-1:24655} state up:standby seq 1 addr [v2:172.21.15.123:6836/2688429163,v1:172.21.15.123:6837/2688429163] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.465 INFO:teuthology.orchestra.run.smithi016.stdout:[mds.i{-1:24664} state up:standby seq 1 addr [v2:172.21.15.123:6838/1332368204,v1:172.21.15.123:6839/1332368204] compat {c=[1],r=[1],i=[1fff]}] 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 msgr2=0x7f63c00b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 0x7f63c00b8b80 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f63c402aa00 tx=0x7f63c4005a20 comp rx=0 tx=0).stop 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 msgr2=0x7f63d419a470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f63d000bd30 tx=0x7f63d0009610 comp rx=0 tx=0).stop 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 shutdown_connections 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63c00b6670 0x7f63c00b8b80 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f63d4109df0 0x7f63d419fcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f63d41094a0 0x7f63d419aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 --2- 172.21.15.16:0/1690936670 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f63d4103b60 0x7f63d419a470 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:51.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 >> 172.21.15.16:0/1690936670 conn(0x7f63d407b850 msgr2=0x7f63d4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:51.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 shutdown_connections 2024-09-21T02:59:51.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:51.463+0000 7f63dc45d640 1 -- 172.21.15.16:0/1690936670 wait complete. 2024-09-21T02:59:52.102 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- bash -c 'ceph osd dump' 2024-09-21T02:59:52.358 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:52.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:52 smithi123 ceph-mon[34645]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:52.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:52 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1690936670' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:59:52.478 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:52 smithi016 ceph-mon[31253]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:52.479 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:52 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1690936670' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:59:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:52 smithi018 ceph-mon[34587]: pgmap v304: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T02:59:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:52 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1690936670' entity='client.admin' cmd={"prefix": "fs dump"} : dispatch 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 -- 172.21.15.16:0/128500949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 msgr2=0x7fd314103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 --2- 172.21.15.16:0/128500949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd314103f70 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7fd304009900 tx=0x7fd30402a410 comp rx=0 tx=0).stop 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 -- 172.21.15.16:0/128500949 shutdown_connections 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 --2- 172.21.15.16:0/128500949 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd314109df0 0x7fd314110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 --2- 172.21.15.16:0/128500949 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd3141094a0 0x7fd3141098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 --2- 172.21.15.16:0/128500949 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd314103f70 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 -- 172.21.15.16:0/128500949 >> 172.21.15.16:0/128500949 conn(0x7fd31407b850 msgr2=0x7fd31407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 -- 172.21.15.16:0/128500949 shutdown_connections 2024-09-21T02:59:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.852+0000 7fd318f0a640 1 -- 172.21.15.16:0/128500949 wait complete. 2024-09-21T02:59:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 Processor -- start 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 -- start start 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 0x7fd31419aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 0x7fd31419fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fd314113b40 con 0x7fd314103b60 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fd3141139c0 con 0x7fd3141094a0 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd318f0a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fd314113cc0 con 0x7fd314109df0 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd31259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.853+0000 7fd31259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34052/0 (socket says 172.21.15.16:34052) 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 -- 172.21.15.16:0/892192078 learned_addr learned my addr 172.21.15.16:0/892192078 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd311d9b640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 0x7fd31419aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd312d9d640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 0x7fd31419fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 -- 172.21.15.16:0/892192078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 msgr2=0x7fd31419fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 0x7fd31419fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 -- 172.21.15.16:0/892192078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 msgr2=0x7fd31419aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 0x7fd31419aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 -- 172.21.15.16:0/892192078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3141a03e0 con 0x7fd314103b60 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd311d9b640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 0x7fd31419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd312d9d640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 0x7fd31419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd31259c640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd304005ab0 tx=0x7fd304003d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd30403f070 con 0x7fd314103b60 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd304031d60 con 0x7fd314103b60 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3141a0670 con 0x7fd314103b60 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd304035030 con 0x7fd314103b60 2024-09-21T02:59:52.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.854+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3141a6ef0 con 0x7fd314103b60 2024-09-21T02:59:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.856+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd314103f70 con 0x7fd314103b60 2024-09-21T02:59:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.857+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7fd304042070 con 0x7fd314103b60 2024-09-21T02:59:52.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.858+0000 7fd3037fe640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 0x7fd2fc0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.858+0000 7fd311d9b640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 0x7fd2fc0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.858+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(77..77 src has 1..77) ==== 8463+0+0 (secure 0 0 0) 0x7fd304108070 con 0x7fd314103b60 2024-09-21T02:59:52.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.859+0000 7fd311d9b640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 0x7fd2fc0b8bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3141091a0 tx=0x7fd308008040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:52.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:52.863+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fd3040ce660 con 0x7fd314103b60 2024-09-21T02:59:53.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.053+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7fd31410bbc0 con 0x7fd314103b60 2024-09-21T02:59:53.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.054+0000 7fd3037fe640 1 -- 172.21.15.16:0/892192078 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v77) ==== 56+0+5689 (secure 0 0 0) 0x7fd3040d2a40 con 0x7fd314103b60 2024-09-21T02:59:53.055 INFO:teuthology.orchestra.run.smithi016.stdout:epoch 77 2024-09-21T02:59:53.055 INFO:teuthology.orchestra.run.smithi016.stdout:fsid ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T02:59:53.055 INFO:teuthology.orchestra.run.smithi016.stdout:created 2024-09-21T02:50:00.888391+0000 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:modified 2024-09-21T02:58:54.732993+0000 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:crush_version 26 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:full_ratio 0.95 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:backfillfull_ratio 0.9 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:nearfull_ratio 0.85 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:require_min_compat_client luminous 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:min_compat_client luminous 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:require_osd_release squid 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:stretch_mode_enabled false 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 23 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 75 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.9 application cephfs read_balance_score 1.50 2024-09-21T02:59:53.056 INFO:teuthology.orchestra.run.smithi016.stdout:max_osd 12 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.0 up in weight 1 up_from 9 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6802/1882705397,v1:172.21.15.16:6803/1882705397] [v2:172.21.15.16:6804/1882705397,v1:172.21.15.16:6805/1882705397] exists,up c38a1c84-7e88-4346-bb53-088922f08bbe 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.1 up in weight 1 up_from 14 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6800/1950134733,v1:172.21.15.18:6801/1950134733] [v2:172.21.15.18:6802/1950134733,v1:172.21.15.18:6803/1950134733] exists,up ab60e425-6410-41cb-a003-2dc7e21c8cb0 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.2 up in weight 1 up_from 19 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6800/3497511537,v1:172.21.15.123:6801/3497511537] [v2:172.21.15.123:6802/3497511537,v1:172.21.15.123:6803/3497511537] exists,up 16bdfeb8-142c-4773-915d-037f7d9f90f7 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.3 up in weight 1 up_from 26 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6810/3760097578,v1:172.21.15.16:6811/3760097578] [v2:172.21.15.16:6812/3760097578,v1:172.21.15.16:6813/3760097578] exists,up 8bdffca0-2cd0-41f5-be85-ebe805e4a696 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.4 up in weight 1 up_from 32 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6808/2331184101,v1:172.21.15.18:6809/2331184101] [v2:172.21.15.18:6810/2331184101,v1:172.21.15.18:6811/2331184101] exists,up 3ee9ba63-3aca-4171-b196-a3b9e633a6b0 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.5 up in weight 1 up_from 38 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6808/2715140085,v1:172.21.15.123:6809/2715140085] [v2:172.21.15.123:6810/2715140085,v1:172.21.15.123:6811/2715140085] exists,up ff1034fc-1839-47d8-80c6-93e0271bd1ec 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.6 up in weight 1 up_from 43 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6818/3190074295,v1:172.21.15.16:6819/3190074295] [v2:172.21.15.16:6820/3190074295,v1:172.21.15.16:6821/3190074295] exists,up c0460abd-d09c-42d4-90e1-fb8ce20982f3 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.7 up in weight 1 up_from 48 up_thru 73 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6816/3027010231,v1:172.21.15.18:6817/3027010231] [v2:172.21.15.18:6818/3027010231,v1:172.21.15.18:6819/3027010231] exists,up 1197b70e-2896-4524-8174-10c95b6b03c7 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.8 up in weight 1 up_from 54 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6816/1022177743,v1:172.21.15.123:6817/1022177743] [v2:172.21.15.123:6818/1022177743,v1:172.21.15.123:6819/1022177743] exists,up df052aaa-1464-46d8-828f-99ce714b76c6 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.9 up in weight 1 up_from 59 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.16:6826/661150471,v1:172.21.15.16:6827/661150471] [v2:172.21.15.16:6828/661150471,v1:172.21.15.16:6829/661150471] exists,up 9d9f79be-daa6-4305-8037-4487eb7b3e5a 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.10 up in weight 1 up_from 64 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.18:6824/1894161437,v1:172.21.15.18:6825/1894161437] [v2:172.21.15.18:6826/1894161437,v1:172.21.15.18:6827/1894161437] exists,up 84176f3a-eba3-4fa1-aaf6-523a10b9d713 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:osd.11 up in weight 1 up_from 69 up_thru 76 down_at 0 last_clean_interval [0,0) [v2:172.21.15.123:6824/4172042931,v1:172.21.15.123:6825/4172042931] [v2:172.21.15.123:6826/4172042931,v1:172.21.15.123:6827/4172042931] exists,up 340835c3-2eab-478a-9def-47542b5fc7bc 2024-09-21T02:59:53.057 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.b [2,1] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1a [8,4] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 2.1d [2,4,8,11] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.0 [5,9] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.6 [4,6] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.17 [4,8] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.1e [3,9] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.24 [4,9] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:pg_upmap_items 3.28 [3,8] 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/2582521578 expires 2024-09-22T02:51:53.053549+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/3267035505 expires 2024-09-22T02:50:23.451106+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/3267035505 expires 2024-09-22T02:50:23.451106+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/3734474829 expires 2024-09-22T02:50:50.132929+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3058825331 expires 2024-09-22T02:51:53.053549+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/156976776 expires 2024-09-22T02:50:23.451106+0000 2024-09-21T02:59:53.058 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6800/2582521578 expires 2024-09-22T02:51:53.053549+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/827779984 expires 2024-09-22T02:50:23.451106+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3317207057 expires 2024-09-22T02:50:50.132929+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1099561653 expires 2024-09-22T02:50:50.132929+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/3096378034 expires 2024-09-22T02:50:23.451106+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:6801/3734474829 expires 2024-09-22T02:50:50.132929+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/1180697074 expires 2024-09-22T02:50:50.132929+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/338428959 expires 2024-09-22T02:51:53.053549+0000 2024-09-21T02:59:53.059 INFO:teuthology.orchestra.run.smithi016.stdout:blocklist 172.21.15.16:0/2917113064 expires 2024-09-22T02:51:53.053549+0000 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 msgr2=0x7fd2fc0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 0x7fd2fc0b8bb0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd3141091a0 tx=0x7fd308008040 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 msgr2=0x7fd31419a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7fd304005ab0 tx=0x7fd304003d50 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 shutdown_connections 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fd2fc0b66a0 0x7fd2fc0b8bb0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd314109df0 0x7fd31419fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fd3141094a0 0x7fd31419aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 --2- 172.21.15.16:0/892192078 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fd314103b60 0x7fd31419a530 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:53.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 >> 172.21.15.16:0/892192078 conn(0x7fd31407b850 msgr2=0x7fd314107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:53.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 shutdown_connections 2024-09-21T02:59:53.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:53.056+0000 7fd318f0a640 1 -- 172.21.15.16:0/892192078 wait complete. 2024-09-21T02:59:53.484 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:53 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/892192078' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:59:53.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:53 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/892192078' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:59:53.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:53 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/892192078' entity='client.admin' cmd={"prefix": "osd dump"} : dispatch 2024-09-21T02:59:53.725 INFO:teuthology.run_tasks:Running task fs.ready... 2024-09-21T02:59:53.742 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:53.999 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:54.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:54 smithi016 ceph-mon[31253]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:54.371 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:54 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]} : dispatch 2024-09-21T02:59:54.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- 172.21.15.16:0/2778362124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8500109df0 msgr2=0x7f8500110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 --2- 172.21.15.16:0/2778362124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8500109df0 0x7f8500110620 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f84f8009ca0 tx=0x7f84f802a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- 172.21.15.16:0/2778362124 shutdown_connections 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 --2- 172.21.15.16:0/2778362124 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8500109df0 0x7f8500110620 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 --2- 172.21.15.16:0/2778362124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f85001094a0 0x7f85001098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 --2- 172.21.15.16:0/2778362124 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 0x7f8500103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- 172.21.15.16:0/2778362124 >> 172.21.15.16:0/2778362124 conn(0x7f850007b850 msgr2=0x7f850007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- 172.21.15.16:0/2778362124 shutdown_connections 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- 172.21.15.16:0/2778362124 wait complete. 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 Processor -- start 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.458+0000 7f850627b640 1 -- start start 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 0x7f850019a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500109df0 0x7f850019fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113ab0 con 0x7f85001094a0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113930 con 0x7f8500103b60 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f850627b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8500113c30 con 0x7f8500109df0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:32988/0 (socket says 172.21.15.16:32988) 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 -- 172.21.15.16:0/790888633 learned_addr learned my addr 172.21.15.16:0/790888633 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84fffff640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 0x7f850019a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f8504818640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500109df0 0x7f850019fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 -- 172.21.15.16:0/790888633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500109df0 msgr2=0x7f850019fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500109df0 0x7f850019fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 -- 172.21.15.16:0/790888633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 msgr2=0x7f850019a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 0x7f850019a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.459+0000 7f84ff7fe640 1 -- 172.21.15.16:0/790888633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85001a03b0 con 0x7f85001094a0 2024-09-21T02:59:54.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f84ff7fe640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84f400d740 tx=0x7f84f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f4017070 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84f400de50 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84f400f620 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85001a06a0 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.460+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85001a6e00 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.461+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8500103f70 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.463+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 21) ==== 154580+0+0 (secure 0 0 0) 0x7f84f401a070 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.464+0000 7f84fd7fa640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 0x7f84e80b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.464+0000 7f84fffff640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 0x7f84e80b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.464+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(78..78 src has 1..78) ==== 8492+0+0 (secure 0 0 0) 0x7f84f40e6e80 con 0x7f85001094a0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.464+0000 7f84fffff640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 0x7f84e80b8b80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84f000bbe0 tx=0x7f84f000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:54.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.467+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f84f40b1910 con 0x7f85001094a0 2024-09-21T02:59:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:54 smithi018 ceph-mon[34587]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:54 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]} : dispatch 2024-09-21T02:59:54.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.693+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f850019b570 con 0x7f85001094a0 2024-09-21T02:59:54.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.693+0000 7f84fd7fa640 1 -- 172.21.15.16:0/790888633 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f850019b570 con 0x7f85001094a0 2024-09-21T02:59:54.695 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:54.695 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:59:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.695+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 msgr2=0x7f84e80b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.695+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 0x7f84e80b8b80 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f84f000bbe0 tx=0x7f84f000b390 comp rx=0 tx=0).stop 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.695+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 msgr2=0x7f850019a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.695+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f84f400d740 tx=0x7f84f400dc00 comp rx=0 tx=0).stop 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 shutdown_connections 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f84e80b6670 0x7f84e80b8b80 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f85001094a0 0x7f850019a9e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8500109df0 0x7f850019fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 --2- 172.21.15.16:0/790888633 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8500103b60 0x7f850019a440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 >> 172.21.15.16:0/790888633 conn(0x7f850007b850 msgr2=0x7f8500105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 shutdown_connections 2024-09-21T02:59:54.698 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:54.696+0000 7f850627b640 1 -- 172.21.15.16:0/790888633 wait complete. 2024-09-21T02:59:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:54 smithi123 ceph-mon[34645]: pgmap v305: 97 pgs: 97 active+clean; 643 KiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T02:59:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:54 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]} : dispatch 2024-09-21T02:59:55.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:55 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-09-21T02:59:55.355 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:55 smithi016 ceph-mon[31253]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:59:55.356 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:55 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/790888633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:55.387 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T02:59:55.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:55.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:55 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-09-21T02:59:55.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:55 smithi018 ceph-mon[34587]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:59:55.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:55 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/790888633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:55.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:55 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.18", "id": [5, 11]}]': finished 2024-09-21T02:59:55.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:55 smithi123 ceph-mon[34645]: osdmap e78: 12 total, 12 up, 12 in 2024-09-21T02:59:55.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:55 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/790888633' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:56.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 -- 172.21.15.16:0/632579718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 msgr2=0x7f019c110630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 --2- 172.21.15.16:0/632579718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c110630 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f0194009ca0 tx=0x7f019402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 -- 172.21.15.16:0/632579718 shutdown_connections 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 --2- 172.21.15.16:0/632579718 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c110630 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 --2- 172.21.15.16:0/632579718 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 --2- 172.21.15.16:0/632579718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 0x7f019c103f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 -- 172.21.15.16:0/632579718 >> 172.21.15.16:0/632579718 conn(0x7f019c0ff5e0 msgr2=0x7f019c101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 -- 172.21.15.16:0/632579718 shutdown_connections 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.142+0000 7f01a2742640 1 -- 172.21.15.16:0/632579718 wait complete. 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.143+0000 7f01a2742640 1 Processor -- start 2024-09-21T02:59:56.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.143+0000 7f01a2742640 1 -- start start 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.143+0000 7f01a2742640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 0x7f019c19a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.143+0000 7f01a2742640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.143+0000 7f01a2742640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c19fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f01a2742640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f019c113ac0 con 0x7f019c109e00 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f01a2742640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f019c113940 con 0x7f019c1094b0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f01a2742640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f019c113c40 con 0x7f019c103b70 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f01a0cdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c19fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55232/0 (socket says 172.21.15.16:55232) 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f01a0cdf640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c19fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33006/0 (socket says 172.21.15.16:33006) 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 -- 172.21.15.16:0/3511084915 learned_addr learned my addr 172.21.15.16:0/3511084915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019bfff640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 0x7f019c19a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 -- 172.21.15.16:0/3511084915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 msgr2=0x7f019c19a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 0x7f019c19a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 -- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 msgr2=0x7f019c19fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c19fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 -- 172.21.15.16:0/3511084915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f019c1a0420 con 0x7f019c1094b0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.144+0000 7f019b7fe640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.145+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f019000ccc0 con 0x7f019c1094b0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.145+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f019c1a0710 con 0x7f019c1094b0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.145+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01900108e0 con 0x7f019c1094b0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.145+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f019c1a6e70 con 0x7f019c1094b0 2024-09-21T02:59:56.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.145+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190018860 con 0x7f019c1094b0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.146+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f019c103f80 con 0x7f019c1094b0 2024-09-21T02:59:56.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.148+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0190010430 con 0x7f019c1094b0 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.148+0000 7f01997fa640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 0x7f01840b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.149+0000 7f019bfff640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 0x7f01840b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.149+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01900e7ba0 con 0x7f019c1094b0 2024-09-21T02:59:56.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.149+0000 7f019bfff640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 0x7f01840b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018c005d00 tx=0x7f018c00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:56.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.152+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01900b26b0 con 0x7f019c1094b0 2024-09-21T02:59:56.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:56 smithi016 ceph-mon[31253]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-21T02:59:56.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:56 smithi016 ceph-mon[31253]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:59:56.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:56 smithi016 ceph-mon[31253]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T02:59:56.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.384+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f019c10bc20 con 0x7f019c1094b0 2024-09-21T02:59:56.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.385+0000 7f01997fa640 1 -- 172.21.15.16:0/3511084915 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f01900b75f0 con 0x7f019c1094b0 2024-09-21T02:59:56.387 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:56.387 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T02:59:56.387 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 msgr2=0x7f01840b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 0x7f01840b8a70 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f018c005d00 tx=0x7f018c00b410 comp rx=0 tx=0).stop 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 msgr2=0x7f019c19aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f019000e9f0 tx=0x7f019000eeb0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 shutdown_connections 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01840b6560 0x7f01840b8a70 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f019c109e00 0x7f019c19fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019c1094b0 0x7f019c19aa50 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 --2- 172.21.15.16:0/3511084915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019c103b70 0x7f019c19a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 >> 172.21.15.16:0/3511084915 conn(0x7f019c0ff5e0 msgr2=0x7f019c101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 shutdown_connections 2024-09-21T02:59:56.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:56.387+0000 7f01a2742640 1 -- 172.21.15.16:0/3511084915 wait complete. 2024-09-21T02:59:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:56 smithi018 ceph-mon[34587]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-21T02:59:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:56 smithi018 ceph-mon[34587]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:59:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:56 smithi018 ceph-mon[34587]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T02:59:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:56 smithi123 ceph-mon[34645]: pgmap v307: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 307 B/s wr, 2 op/s 2024-09-21T02:59:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:56 smithi123 ceph-mon[34645]: osdmap e79: 12 total, 12 up, 12 in 2024-09-21T02:59:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:56 smithi123 ceph-mon[34645]: mgrmap e22: x(active, since 8m), standbys: z, y 2024-09-21T02:59:57.100 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T02:59:57.358 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:57 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3511084915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:57.368 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:57.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:57 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3511084915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:57.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:57 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3511084915' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 -- 172.21.15.16:0/2738517251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c102ae0 msgr2=0x7f4c8c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 --2- 172.21.15.16:0/2738517251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c102ef0 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009900 tx=0x7f4c7c02a410 comp rx=0 tx=0).stop 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 -- 172.21.15.16:0/2738517251 shutdown_connections 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 --2- 172.21.15.16:0/2738517251 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c108d20 0x7f4c8c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 --2- 172.21.15.16:0/2738517251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 0x7f4c8c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 --2- 172.21.15.16:0/2738517251 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c102ef0 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 -- 172.21.15.16:0/2738517251 >> 172.21.15.16:0/2738517251 conn(0x7f4c8c079760 msgr2=0x7f4c8c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:57.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 -- 172.21.15.16:0/2738517251 shutdown_connections 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 -- 172.21.15.16:0/2738517251 wait complete. 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.842+0000 7f4c912ce640 1 Processor -- start 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 -- start start 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c19a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 0x7f4c8c19aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113ab0 con 0x7f4c8c108d20 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113930 con 0x7f4c8c102ae0 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c912ce640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c113c30 con 0x7f4c8c1083d0 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33024/0 (socket says 172.21.15.16:33024) 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 -- 172.21.15.16:0/954410128 learned_addr learned my addr 172.21.15.16:0/954410128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8affd640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c19a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:57.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8a7fc640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 0x7f4c8c19aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 -- 172.21.15.16:0/954410128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 msgr2=0x7f4c8c19aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 0x7f4c8c19aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 -- 172.21.15.16:0/954410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c102ae0 msgr2=0x7f4c8c19a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c19a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.843+0000 7f4c8b7fe640 1 -- 172.21.15.16:0/954410128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c1a03d0 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c8b7fe640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400eaa0 tx=0x7f4c8400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c8400cdb0 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a06c0 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c84005020 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c840116d0 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.844+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a6e20 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.845+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c103d10 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.847+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4c840051d0 con 0x7f4c8c108d20 2024-09-21T02:59:57.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.848+0000 7f4c73fff640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 0x7f4c740b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.848+0000 7f4c8affd640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 0x7f4c740b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.848+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4c840e79e0 con 0x7f4c8c108d20 2024-09-21T02:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.849+0000 7f4c8affd640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 0x7f4c740b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c031440 tx=0x7f4c7c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:57.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:57.852+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c84016030 con 0x7f4c8c108d20 2024-09-21T02:59:58.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.040+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4c8c102ef0 con 0x7f4c8c108d20 2024-09-21T02:59:58.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.041+0000 7f4c73fff640 1 -- 172.21.15.16:0/954410128 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23127 (secure 0 0 0) 0x7f4c8c102ef0 con 0x7f4c8c108d20 2024-09-21T02:59:58.043 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T02:59:58.043 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:59:55.231626+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 msgr2=0x7f4c740b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 0x7f4c740b8a70 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c031440 tx=0x7f4c7c009560 comp rx=0 tx=0).stop 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 msgr2=0x7f4c8c19fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f4c8400eaa0 tx=0x7f4c8400ef60 comp rx=0 tx=0).stop 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 shutdown_connections 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c740b6560 0x7f4c740b8a70 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c8c108d20 0x7f4c8c19fca0 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c8c1083d0 0x7f4c8c19aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 --2- 172.21.15.16:0/954410128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c8c102ae0 0x7f4c8c19a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.043+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 >> 172.21.15.16:0/954410128 conn(0x7f4c8c079760 msgr2=0x7f4c8c101580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.044+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 shutdown_connections 2024-09-21T02:59:58.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:58.044+0000 7f4c912ce640 1 -- 172.21.15.16:0/954410128 wait complete. 2024-09-21T02:59:58.508 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:58 smithi016 ceph-mon[31253]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T02:59:58.509 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 02:59:58 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/954410128' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:58 smithi018 ceph-mon[34587]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T02:59:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 02:59:58 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/954410128' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:58 smithi123 ceph-mon[34645]: pgmap v309: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T02:59:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 02:59:58 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/954410128' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T02:59:58.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 20, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-09-21T02:58:28.365002+0000', 'modified': '2024-09-21T02:58:37.482868+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24701}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24674': {'gid': 24674, 'name': 'h', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.18:6837/2180643175', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6836', 'nonce': 2180643175}, {'type': 'v1', 'addr': '172.21.15.18:6837', 'nonce': 2180643175}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24701': {'gid': 24701, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 4, 'addr': '172.21.15.18:6839/253530783', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.18:6838', 'nonce': 253530783}, {'type': 'v1', 'addr': '172.21.15.18:6839', 'nonce': 253530783}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540719139924082687, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24701, 'qdb_cluster': [24701]} 2024-09-21T02:59:58.729 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-09-21T02:59:58.729 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.b status 2024-09-21T02:59:58.991 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T02:59:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 -- 172.21.15.16:0/3231510887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4102ae0 msgr2=0x7f65f4102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 --2- 172.21.15.16:0/3231510887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4102ae0 0x7f65f4102ef0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f65e4009ca0 tx=0x7f65e402a8c0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 -- 172.21.15.16:0/3231510887 shutdown_connections 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 --2- 172.21.15.16:0/3231510887 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4108d20 0x7f65f410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 --2- 172.21.15.16:0/3231510887 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 0x7f65f41087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 --2- 172.21.15.16:0/3231510887 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4102ae0 0x7f65f4102ef0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 -- 172.21.15.16:0/3231510887 >> 172.21.15.16:0/3231510887 conn(0x7f65f40797b0 msgr2=0x7f65f4079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 -- 172.21.15.16:0/3231510887 shutdown_connections 2024-09-21T02:59:59.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.490+0000 7f65f88da640 1 -- 172.21.15.16:0/3231510887 wait complete. 2024-09-21T02:59:59.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.491+0000 7f65f88da640 1 Processor -- start 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.491+0000 7f65f88da640 1 -- start start 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.491+0000 7f65f88da640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4102ae0 0x7f65f419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.491+0000 7f65f88da640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 0x7f65f419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f88da640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f88da640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4113b40 con 0x7f65f4108d20 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f88da640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f65f41139c0 con 0x7f65f4102ae0 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f88da640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f65f4113cc0 con 0x7f65f41083d0 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33040/0 (socket says 172.21.15.16:33040) 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 -- 172.21.15.16:0/4013226190 learned_addr learned my addr 172.21.15.16:0/4013226190 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f259c640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4102ae0 0x7f65f419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 0x7f65f419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 -- 172.21.15.16:0/4013226190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 msgr2=0x7f65f419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 0x7f65f419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 -- 172.21.15.16:0/4013226190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4102ae0 msgr2=0x7f65f419a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4102ae0 0x7f65f419a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 -- 172.21.15.16:0/4013226190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65f41a03a0 con 0x7f65f4108d20 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65f2d9d640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009fd0 tx=0x7f65ec00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.492+0000 7f65e37fe640 1 -- 172.21.15.16:0/4013226190 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec015020 con 0x7f65f4108d20 2024-09-21T02:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.493+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65f41a0690 con 0x7f65f4108d20 2024-09-21T02:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.493+0000 7f65e37fe640 1 -- 172.21.15.16:0/4013226190 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65ec011ce0 con 0x7f65f4108d20 2024-09-21T02:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.493+0000 7f65e37fe640 1 -- 172.21.15.16:0/4013226190 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65ec010dc0 con 0x7f65f4108d20 2024-09-21T02:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.493+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65f41a6eb0 con 0x7f65f4108d20 2024-09-21T02:59:59.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.495+0000 7f65e37fe640 1 -- 172.21.15.16:0/4013226190 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f65ec018020 con 0x7f65f4108d20 2024-09-21T02:59:59.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.495+0000 7f65fee4e740 1 Processor -- start 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 -- start start 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b4ff00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67810a0 0x564ed6b50440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ed6ac7c10 con 0x564ed6789280 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564ed6ab7040 con 0x564ed67810a0 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65fee4e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ed6abae50 con 0x564ed67a9f60 2024-09-21T02:59:59.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33042/0 (socket says 172.21.15.16:33042) 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 -- 172.21.15.16:0/47501433 learned_addr learned my addr 172.21.15.16:0/47501433 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b4ff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f259c640 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67810a0 0x564ed6b50440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65e37fe640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 0x7f65dc0b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 -- 172.21.15.16:0/47501433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 msgr2=0x564ed6b4ff00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b4ff00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 -- 172.21.15.16:0/47501433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67810a0 msgr2=0x564ed6b50440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67810a0 0x564ed6b50440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.496+0000 7f65f1d9b640 1 -- 172.21.15.16:0/47501433 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed6815540 con 0x564ed6789280 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65f2d9d640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 0x7f65dc0b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011580 tx=0x7f65e800a700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c86b3ca4cf20df4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65e37fe640 1 -- 172.21.15.16:0/4013226190 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f65ec0e6b40 con 0x7f65f4108d20 2024-09-21T02:59:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65e0ff9640 1 -- 172.21.15.16:0/47501433 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8005220 con 0x564ed6789280 2024-09-21T02:59:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65e0ff9640 1 -- 172.21.15.16:0/47501433 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e8004c50 con 0x564ed6789280 2024-09-21T02:59:59.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.497+0000 7f65e0ff9640 1 -- 172.21.15.16:0/47501433 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800f9f0 con 0x564ed6789280 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- 172.21.15.16:0/47501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 msgr2=0x564ed6b50980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65f2d9d640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 0x7f65dc0b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65ec089fb0 tx=0x7f65ec06dcf0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f65e8011580 tx=0x7f65e800a700 comp rx=0 tx=0).stop 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- 172.21.15.16:0/47501433 shutdown_connections 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed6789280 0x564ed6b50980 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67810a0 0x564ed6b50440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 --2- 172.21.15.16:0/47501433 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b4ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- 172.21.15.16:0/47501433 >> 172.21.15.16:0/47501433 conn(0x564ed679e350 msgr2=0x564ed6b4ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- 172.21.15.16:0/47501433 shutdown_connections 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- 172.21.15.16:0/47501433 wait complete. 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.498+0000 7f65fee4e740 1 -- start start 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 Processor -- start 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b6bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ed69185f0 con 0x564ed67810a0 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564ed6a52010 con 0x564ed67a9f60 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b62a40 con 0x564ed6789280 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65fee4e740 10 client.0 _ll_register_callbacks cb 0x564ed6ab1340 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33058/0 (socket says 172.21.15.16:33058) 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65f1d9b640 1 -- 172.21.15.16:0/3368742710 learned_addr learned my addr 172.21.15.16:0/3368742710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65f259c640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.499+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b6bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 -- 172.21.15.16:0/3368742710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 msgr2=0x564ed6b60340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 -- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 msgr2=0x564ed6b6bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b6bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 -- 172.21.15.16:0/3368742710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed6a7fa00 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005770 tx=0x7f65e8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aae6e3e30d5dc51c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800f770 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f65e800f910 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e8004c50 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f65e800fb10 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65fee4e740 10 client.15288 fetch_fsmap learned FSMap version 20 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ed6b6e9a0 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.500+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e8004c50 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f65e8032020 con 0x564ed67810a0 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 10 client.15288 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 10 client.15288 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 20 client.15288 populate_metadata read hostname 'smithi016' 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed67688e0 0x564ed6b720e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 4 client.15288 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T02:59:59.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ed6ac9b60 con 0x564ed67688e0 2024-09-21T02:59:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65f259c640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed67688e0 0x564ed6b720e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65d9ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.501+0000 7f65f259c640 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed67688e0 0x564ed6b720e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7413c575abc3f3c6 server_cookie=76a9d19b335bf21 in_seq=0 out_seq=0 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65d9ffb640 1 -- 172.21.15.16:0/3368742710 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x564ed6ac9b60 con 0x564ed67688e0 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65d9ffb640 10 client.15288 handle_command_reply: tid=0 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 client.15288 shutdown 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 msgr2=0x564ed6b5fe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f65e8005770 tx=0x7f65e8005850 comp rx=0 tx=0).stop 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 shutdown_connections 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed67688e0 0x564ed6b720e0 crc :-1 s=READY pgs=20 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b6bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.502+0000 7f65fee4e740 1 --2- 172.21.15.16:0/3368742710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fe00 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.503+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 >> 172.21.15.16:0/3368742710 conn(0x564ed679e350 msgr2=0x564ed6b518f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.503+0000 7f65d9ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:59.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.503+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 shutdown_connections 2024-09-21T02:59:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.503+0000 7f65fee4e740 1 -- 172.21.15.16:0/3368742710 wait complete. 2024-09-21T02:59:59.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.503+0000 7f65fee4e740 20 client.15288 trim_cache size 0 max 16384 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 Processor -- start 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 -- start start 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b54f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b70420 con 0x564ed67810a0 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b70af0 con 0x564ed67a9f60 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65fee4e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b711c0 con 0x564ed6789280 2024-09-21T02:59:59.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33074/0 (socket says 172.21.15.16:33074) 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1009447978 learned_addr learned my addr 172.21.15.16:0/1009447978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.521+0000 7f65f259c640 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1009447978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 msgr2=0x564ed6b60320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1009447978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 msgr2=0x564ed6b54f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b54f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1009447978 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ed6b52cc0 con 0x564ed67810a0 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f259c640 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T02:59:59.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65e800a650 tx=0x7f65e8005000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dac12db6163f0b22 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65dbfff640 1 -- 172.21.15.16:0/1009447978 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e800a9f0 con 0x564ed67810a0 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65dbfff640 1 -- 172.21.15.16:0/1009447978 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e8039070 con 0x564ed67810a0 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.522+0000 7f65dbfff640 1 -- 172.21.15.16:0/1009447978 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8034910 con 0x564ed67810a0 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- 172.21.15.16:0/1009447978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 msgr2=0x564ed6b5fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f65e800a650 tx=0x7f65e8005000 comp rx=0 tx=0).stop 2024-09-21T02:59:59.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- 172.21.15.16:0/1009447978 shutdown_connections 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed67a9f60 0x564ed6b54f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed6789280 0x564ed6b60320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1009447978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b5fde0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- 172.21.15.16:0/1009447978 >> 172.21.15.16:0/1009447978 conn(0x564ed679e350 msgr2=0x564ed6b55a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- 172.21.15.16:0/1009447978 shutdown_connections 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- 172.21.15.16:0/1009447978 wait complete. 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 -- start start 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.523+0000 7f65fee4e740 1 Processor -- start 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 0x564ed6b75520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b78fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b71f60 con 0x564ed67810a0 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b71890 con 0x564ed6789280 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564ed6b72630 con 0x564ed67a9f60 2024-09-21T02:59:59.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65fee4e740 10 client.0 _ll_register_callbacks cb 0x7f65e40093e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33090/0 (socket says 172.21.15.16:33090) 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1200124706 learned_addr learned my addr 172.21.15.16:0/1200124706 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f259c640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 0x564ed6b75520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1200124706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 msgr2=0x564ed6b78fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b78fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 msgr2=0x564ed6b75520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 0x564ed6b75520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 -- 172.21.15.16:0/1200124706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ed6b79640 con 0x564ed67810a0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f259c640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 0x564ed6b75520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65f1d9b640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65e80054f0 tx=0x7f65e800a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec285e6aebffd04e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.524+0000 7f65d9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f65e8041030 con 0x564ed67810a0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f65e800fcb0 con 0x564ed67810a0 2024-09-21T02:59:59.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f65e8039070 con 0x564ed67810a0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f65e800f690 con 0x564ed67810a0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 10 client.15294 fetch_fsmap learned FSMap version 20 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x564ed6b7aa00 con 0x564ed67810a0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f65e8039070 con 0x564ed67810a0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f65e804d020 con 0x564ed67810a0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 10 client.15294 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 10 client.15294 resolve_mds: resolved name 'b' to daemon mds.b 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 20 client.15294 populate_metadata read hostname 'smithi016' 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed673dc90 0x564ed6be02e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 4 client.15294 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "status"}] 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.525+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "status"}) -- 0x564ed6ac5610 con 0x564ed673dc90 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65f259c640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed673dc90 0x564ed6be02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65d9ffb640 10 client.15294 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:59.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65f259c640 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed673dc90 0x564ed6be02e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=adec67d13f78b501 server_cookie=7e1d3d4a6ad2a0a4 in_seq=0 out_seq=0 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65d9ffb640 1 -- 172.21.15.16:0/1200124706 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ed6ac5610 con 0x564ed673dc90 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65d9ffb640 10 client.15294 handle_command_reply: tid=0 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65fee4e740 1 client.15294 shutdown 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 msgr2=0x564ed6b74f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f65e80054f0 tx=0x7f65e800a2d0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 shutdown_connections 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.526+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x564ed673dc90 0x564ed6be02e0 crc :-1 s=READY pgs=21 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x564ed67a9f60 0x564ed6b78fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x564ed6789280 0x564ed6b75520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 --2- 172.21.15.16:0/1200124706 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x564ed67810a0 0x564ed6b74f60 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65d9ffb640 0 client.15294 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 >> 172.21.15.16:0/1200124706 conn(0x564ed679e350 msgr2=0x564ed6b5ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 shutdown_connections 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 1 -- 172.21.15.16:0/1200124706 wait complete. 2024-09-21T02:59:59.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65fee4e740 20 client.15294 trim_cache size 0 max 16384 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "cluster_fsid": "ffc7910c-77c3-11ef-baf5-efdc52797490", 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "whoami": 0, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "id": 24701, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "want_state": "up:active", 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "state": "up:active", 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "fs_name": "cephfs", 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "rank_uptime": 90.888955784000004, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "mdsmap_epoch": 20, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch": 79, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_epoch_barrier": 75, 2024-09-21T02:59:59.533 INFO:teuthology.orchestra.run.smithi016.stdout: "uptime": 101.74282866199999 2024-09-21T02:59:59.534 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T02:59:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 msgr2=0x7f65dc0b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 0x7f65dc0b8aa0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f65ec089fb0 tx=0x7f65ec06dcf0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 msgr2=0x7f65f419fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T02:59:59.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f65ec009fd0 tx=0x7f65ec00b820 comp rx=0 tx=0).stop 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 shutdown_connections 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65dc0b6590 0x7f65dc0b8aa0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65f4108d20 0x7f65f419fd30 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65f41083d0 0x7f65f419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 --2- 172.21.15.16:0/4013226190 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65f4102ae0 0x7f65f419a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 >> 172.21.15.16:0/4013226190 conn(0x7f65f40797b0 msgr2=0x7f65f4106460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 shutdown_connections 2024-09-21T02:59:59.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T02:59:59.527+0000 7f65f88da640 1 -- 172.21.15.16:0/4013226190 wait complete. 2024-09-21T03:00:00.238 INFO:teuthology.run_tasks:Running task kclient... 2024-09-21T03:00:00.262 INFO:tasks.kclient:Mounting kernel clients... 2024-09-21T03:00:00.262 INFO:tasks.kclient:config is {'client.0': {}} 2024-09-21T03:00:00.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:00.263 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:00:00.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:00.281 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-21T03:00:00.351 INFO:teuthology.orchestra.run.smithi016.stderr:Cannot find device "ceph-brx" 2024-09-21T03:00:00.352 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:00:00.352 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v2', 'mount_subvol_num': 1} 2024-09-21T03:00:00.353 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-09-21T03:00:00.353 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:00.353 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs ls 2024-09-21T03:00:00.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:00 smithi016 ceph-mon[31253]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-21T03:00:00.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:00 smithi016 ceph-mon[31253]: overall HEALTH_OK 2024-09-21T03:00:00.651 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:00 smithi018 ceph-mon[34587]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-21T03:00:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:00 smithi018 ceph-mon[34587]: overall HEALTH_OK 2024-09-21T03:00:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:00 smithi123 ceph-mon[34645]: pgmap v310: 97 pgs: 97 active+clean; 643 KiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s rd, 383 B/s wr, 1 op/s 2024-09-21T03:00:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:00 smithi123 ceph-mon[34645]: overall HEALTH_OK 2024-09-21T03:00:01.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1661017405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 msgr2=0x7f8d841087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1661017405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d841087e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f8d78009900 tx=0x7f8d7802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1661017405 shutdown_connections 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1661017405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d841087e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1661017405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84108d20 0x7f8d8410f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1661017405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d84102ae0 0x7f8d84102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.114+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1661017405 >> 172.21.15.16:0/1661017405 conn(0x7f8d84079760 msgr2=0x7f8d84079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.115+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1661017405 shutdown_connections 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.115+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1661017405 wait complete. 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.115+0000 7f8d89f0c640 1 Processor -- start 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.115+0000 7f8d89f0c640 1 -- start start 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84102ae0 0x7f8d841993c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d84108d20 0x7f8d8419ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:01.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112990 con 0x7f8d841083d0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112810 con 0x7f8d84102ae0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d89f0c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8d84112b10 con 0x7f8d84108d20 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33110/0 (socket says 172.21.15.16:33110) 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 -- 172.21.15.16:0/1548287772 learned_addr learned my addr 172.21.15.16:0/1548287772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d837fe640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84102ae0 0x7f8d841993c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 -- 172.21.15.16:0/1548287772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d84108d20 msgr2=0x7f8d8419ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d84108d20 0x7f8d8419ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 -- 172.21.15.16:0/1548287772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84102ae0 msgr2=0x7f8d841993c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84102ae0 0x7f8d841993c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 -- 172.21.15.16:0/1548287772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d8419f330 con 0x7f8d841083d0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d82ffd640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006010 tx=0x7f8d78004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.116+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78046070 con 0x7f8d841083d0 2024-09-21T03:00:01.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.117+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d780044d0 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.117+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d78035050 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.117+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d8419f5c0 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.117+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d8406bb40 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.118+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8d84103d10 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.119+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f8d78049070 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.120+0000 7f8d80ff9640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 0x7f8d6c0b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.120+0000 7f8d837fe640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 0x7f8d6c0b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.120+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8d781090b0 con 0x7f8d841083d0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.121+0000 7f8d837fe640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 0x7f8d6c0b8af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005fd0 tx=0x7f8d7400b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:01.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.123+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f8d78050080 con 0x7f8d841083d0 2024-09-21T03:00:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.355+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f8d84102ef0 con 0x7f8d841083d0 2024-09-21T03:00:01.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.355+0000 7f8d80ff9640 1 -- 172.21.15.16:0/1548287772 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v20) ==== 53+0+73 (secure 0 0 0) 0x7f8d84102ef0 con 0x7f8d841083d0 2024-09-21T03:00:01.357 INFO:teuthology.orchestra.run.smithi016.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2024-09-21T03:00:01.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 msgr2=0x7f8d6c0b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 0x7f8d6c0b8af0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8d74005fd0 tx=0x7f8d7400b390 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 msgr2=0x7f8d84199960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f8d78006010 tx=0x7f8d78004060 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 shutdown_connections 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8d6c0b65e0 0x7f8d6c0b8af0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8d841083d0 0x7f8d84199960 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8d84108d20 0x7f8d8419ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 --2- 172.21.15.16:0/1548287772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8d84102ae0 0x7f8d841993c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 >> 172.21.15.16:0/1548287772 conn(0x7f8d84079760 msgr2=0x7f8d84104440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 shutdown_connections 2024-09-21T03:00:01.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:01.358+0000 7f8d89f0c640 1 -- 172.21.15.16:0/1548287772 wait complete. 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi016.front.sepia.ceph.com 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-09-21T03:00:02.071 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-09-21T03:00:02.072 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-09-21T03:00:02.072 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:00:02.072 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:02.072 DEBUG:teuthology.orchestra.run.smithi016:> ip addr 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: inet 127.0.0.1/8 scope host lo 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 ::1/128 scope host 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e0 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f0 2024-09-21T03:00:02.091 INFO:teuthology.orchestra.run.smithi016.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:6b:fd:e1 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp7s0f1 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:64 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f0 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: inet 172.21.15.16/20 brd 172.21.15.255 scope global dynamic noprefixroute ens1f0 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft 42162sec preferred_lft 42162sec 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: inet6 fe80::ec4:7aff:febd:1564/64 scope link 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: valid_lft forever preferred_lft forever 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout:5: ens1f1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: link/ether 0c:c4:7a:bd:15:65 brd ff:ff:ff:ff:ff:ff 2024-09-21T03:00:02.092 INFO:teuthology.orchestra.run.smithi016.stdout: altname enp1s0f1 2024-09-21T03:00:02.093 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add name ceph-brx type bridge 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr flush dev ceph-brx 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set ceph-brx up 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-09-21T03:00:02.093 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:00:02.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:02.291 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:02 smithi016 ceph-mon[31253]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T03:00:02.292 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1548287772' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:00:02.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:02.325 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:02.325 DEBUG:teuthology.orchestra.run.smithi016:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-09-21T03:00:02.351 INFO:teuthology.orchestra.run.smithi016.stdout:1 2024-09-21T03:00:02.353 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:02.353 DEBUG:teuthology.orchestra.run.smithi016:> route 2024-09-21T03:00:02.424 INFO:teuthology.orchestra.run.smithi016.stdout:Kernel IP routing table 2024-09-21T03:00:02.424 INFO:teuthology.orchestra.run.smithi016.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-09-21T03:00:02.424 INFO:teuthology.orchestra.run.smithi016.stdout:default _gateway 0.0.0.0 UG 100 0 0 ens1f0 2024-09-21T03:00:02.425 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 ens1f0 2024-09-21T03:00:02.425 INFO:teuthology.orchestra.run.smithi016.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-09-21T03:00:02.425 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:00:02.425 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:00:02.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2024-09-21T03:00:02.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2024-09-21T03:00:02.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2024-09-21T03:00:02.426 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:00:02.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:02 smithi018 ceph-mon[34587]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T03:00:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:02 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1548287772' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:00:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:02 smithi123 ceph-mon[34645]: pgmap v311: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 383 B/s wr, 3 op/s 2024-09-21T03:00:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:02 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1548287772' entity='client.admin' cmd={"prefix": "fs ls"} : dispatch 2024-09-21T03:00:02.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:02.738 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:02.738 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:00:02.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:02.754 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list-id 2024-09-21T03:00:02.810 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:00:02.810 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:00:02.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:00:02.810 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-09-21T03:00:02.811 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:00:02.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:02.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:02.931 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-09-21T03:00:02.931 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:00:02.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:02 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.072 DEBUG:teuthology.orchestra.run.smithi016:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-09-21T03:00:03.072 DEBUG:teuthology.orchestra.run.smithi016:> set -e 2024-09-21T03:00:03.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set brx.0 up 2024-09-21T03:00:03.072 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link set dev brx.0 master ceph-brx 2024-09-21T03:00:03.072 DEBUG:teuthology.orchestra.run.smithi016:> ') 2024-09-21T03:00:03.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:03.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.153 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:00:03.154 DEBUG:teuthology.orchestra.run.smithi016:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:03.169 INFO:teuthology.orchestra.run.smithi016.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-09-21T03:00:03.170 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-09-21T03:00:03.170 DEBUG:teuthology.orchestra.run.smithi016:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:03.226 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-09-21T03:00:03.226 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-09-21T03:00:03.226 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/volumes/qa/sv_1/1bf5d314-c769-4152-adce-e93b2c8ff91a 2024-09-21T03:00:03.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:03.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/volumes/qa/sv_1/1bf5d314-c769-4152-adce-e93b2c8ff91a /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:00:03.339 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:00:03.340 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T03:00:03.478 INFO:teuthology.orchestra.run.smithi016.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,ms_mode=secure,wsync,nofallback 2024-09-21T03:00:03.478 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync". 2024-09-21T03:00:03.478 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: resolved to: "172.21.15.16:3300,172.21.15.18:3300,172.21.15.123:3300" 2024-09-21T03:00:03.478 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: trying mount with new device syntax: 0@ffc7910c-77c3-11ef-baf5-efdc52797490.cephfs=/volumes/qa/sv_1/1bf5d314-c769-4152-adce-e93b2c8ff91a 2024-09-21T03:00:03.478 INFO:teuthology.orchestra.run.smithi016.stdout:mount.ceph: options "norequire_active_mds,norbytes,ms_mode=secure,wsync,name=0,key=0,mon_addr=172.21.15.16:3300/172.21.15.18:3300/172.21.15.123:3300" will pass to kernel 2024-09-21T03:00:03.479 INFO:teuthology.orchestra.run.smithi016.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-09-21T03:00:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: You just mounted a file system that supports labels which does not 2024-09-21T03:00:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-09-21T03:00:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: applications will generate AVC messages and not be allowed access to 2024-09-21T03:00:03.479 INFO:teuthology.orchestra.run.smithi016.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-09-21T03:00:03.482 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-09-21T03:00:03.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:00:03.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:03.510 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:00:03.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:00:03.603 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.609 INFO:teuthology.orchestra.run.smithi016.stdout:client15303 2024-09-21T03:00:03.610 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:00:03.610 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:00:03.627 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:00:03.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:00:03.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.724 INFO:teuthology.orchestra.run.smithi016.stdout:client15303 2024-09-21T03:00:03.725 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-09-21T03:00:03.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-09-21T03:00:03.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.784 INFO:teuthology.orchestra.run.smithi016.stdout:client15303 2024-09-21T03:00:03.785 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490.client15303/status') 2024-09-21T03:00:03.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-09-21T03:00:03.844 INFO:teuthology.orchestra.run.smithi016.stdout:instance: client.15303 (3)192.168.0.1:0/3262895334 2024-09-21T03:00:03.844 INFO:teuthology.orchestra.run.smithi016.stdout:blocklisted: false 2024-09-21T03:00:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records in 2024-09-21T03:00:03.844 INFO:teuthology.orchestra.run.smithi016.stderr:0+1 records out 2024-09-21T03:00:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:70 bytes copied, 2.3201e-05 s, 3.0 MB/s 2024-09-21T03:00:03.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:03 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-09-21T03:00:03.852 INFO:teuthology.run_tasks:Running task check-counter... 2024-09-21T03:00:03.863 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-09-21T03:00:03.918 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 -- 172.21.15.16:0/3673636076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc105260 msgr2=0x7f27cc10d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3673636076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc105260 0x7f27cc10d7a0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009900 tx=0x7f27bc02a410 comp rx=0 tx=0).stop 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 -- 172.21.15.16:0/3673636076 shutdown_connections 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3673636076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc105260 0x7f27cc10d7a0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3673636076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc10dd70 0x7f27cc110260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3673636076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc104910 0x7f27cc104d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 -- 172.21.15.16:0/3673636076 >> 172.21.15.16:0/3673636076 conn(0x7f27cc0fe3c0 msgr2=0x7f27cc100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 -- 172.21.15.16:0/3673636076 shutdown_connections 2024-09-21T03:00:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.031+0000 7f27d47a1640 1 -- 172.21.15.16:0/3673636076 wait complete. 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 Processor -- start 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 -- start start 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc105260 0x7f27cc19a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc10dd70 0x7f27cc19fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc113990 con 0x7f27cc104910 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.032+0000 7f27d47a1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc113810 con 0x7f27cc105260 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d47a1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27cc113b10 con 0x7f27cc10dd70 2024-09-21T03:00:04.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33120/0 (socket says 172.21.15.16:33120) 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 -- 172.21.15.16:0/3315801341 learned_addr learned my addr 172.21.15.16:0/3315801341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d1d3c640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc105260 0x7f27cc19a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d2d3e640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc10dd70 0x7f27cc19fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 -- 172.21.15.16:0/3315801341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc10dd70 msgr2=0x7f27cc19fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc10dd70 0x7f27cc19fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 -- 172.21.15.16:0/3315801341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc105260 msgr2=0x7f27cc19a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc105260 0x7f27cc19a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 -- 172.21.15.16:0/3315801341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27cc1a0340 con 0x7f27cc104910 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d253d640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f27c800d950 tx=0x7f27c800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c8017030 con 0x7f27cc104910 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27cc1a0630 con 0x7f27cc104910 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27c8004020 con 0x7f27cc104910 2024-09-21T03:00:04.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f27c80047a0 con 0x7f27cc104910 2024-09-21T03:00:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.033+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27cc1a6dc0 con 0x7f27cc104910 2024-09-21T03:00:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.035+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f27cc06c010 con 0x7f27cc104910 2024-09-21T03:00:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.036+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f27c8009a30 con 0x7f27cc104910 2024-09-21T03:00:04.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.037+0000 7f27c37fe640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 0x7f27b80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.037+0000 7f27d1d3c640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 0x7f27b80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.037+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f27c80e6ef0 con 0x7f27cc104910 2024-09-21T03:00:04.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.037+0000 7f27d1d3c640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 0x7f27b80b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009230 tx=0x7f27bc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.041+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f27c80ad680 con 0x7f27cc104910 2024-09-21T03:00:04.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.233+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f27cc1a70d0 con 0x7f27cc104910 2024-09-21T03:00:04.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.235+0000 7f27c37fe640 1 -- 172.21.15.16:0/3315801341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23127 (secure 0 0 0) 0x7f27c80b1a60 con 0x7f27cc104910 2024-09-21T03:00:04.237 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:04.237 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:59:55.231626+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:00:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 msgr2=0x7f27b80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 0x7f27b80b8a70 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f27bc009230 tx=0x7f27bc031400 comp rx=0 tx=0).stop 2024-09-21T03:00:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 msgr2=0x7f27cc19a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f27c800d950 tx=0x7f27c800de10 comp rx=0 tx=0).stop 2024-09-21T03:00:04.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 shutdown_connections 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27b80b6560 0x7f27b80b8a70 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27cc10dd70 0x7f27cc19fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27cc105260 0x7f27cc19a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 --2- 172.21.15.16:0/3315801341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27cc104910 0x7f27cc19a410 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 >> 172.21.15.16:0/3315801341 conn(0x7f27cc0fe3c0 msgr2=0x7f27cc106770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 shutdown_connections 2024-09-21T03:00:04.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.237+0000 7f27d47a1640 1 -- 172.21.15.16:0/3315801341 wait complete. 2024-09-21T03:00:04.251 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:53:55.521347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:58:24.062466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:58:26.393380+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:00:04.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:00:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.367+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1176622720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108c50 msgr2=0x7f3fcc10f480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.367+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1176622720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108c50 0x7f3fcc10f480 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f3fc4009900 tx=0x7f3fc402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:04.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.367+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1176622720 shutdown_connections 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1176622720 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108c50 0x7f3fcc10f480 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1176622720 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc108300 0x7f3fcc108710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1176622720 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc102a20 0x7f3fcc102e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1176622720 >> 172.21.15.16:0/1176622720 conn(0x7f3fcc079630 msgr2=0x7f3fcc079a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1176622720 shutdown_connections 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1176622720 wait complete. 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 Processor -- start 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 -- start start 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.368+0000 7f3fd30b6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 0x7f3fcc19a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc108c50 0x7f3fcc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc113a30 con 0x7f3fcc108300 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc1138b0 con 0x7f3fcc108c50 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3fcc113bb0 con 0x7f3fcc102a20 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49540/0 (socket says 172.21.15.16:49540) 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 -- 172.21.15.16:0/1914574337 learned_addr learned my addr 172.21.15.16:0/1914574337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd0e52640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 0x7f3fcc19a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd1653640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc108c50 0x7f3fcc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 -- 172.21.15.16:0/1914574337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 msgr2=0x7f3fcc19a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 0x7f3fcc19a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.373 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 -- 172.21.15.16:0/1914574337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc108c50 msgr2=0x7f3fcc19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc108c50 0x7f3fcc19fba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 -- 172.21.15.16:0/1914574337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fcc1a02a0 con 0x7f3fcc108300 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd0e52640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 0x7f3fcc19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fcbfff640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00ebc0 tx=0x7f3fbc00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc010630 con 0x7f3fcc108300 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fbc010c80 con 0x7f3fcc108300 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fcc1a0590 con 0x7f3fcc108300 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.369+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fbc017ba0 con 0x7f3fcc108300 2024-09-21T03:00:04.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.370+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fcc1a6d20 con 0x7f3fcc108300 2024-09-21T03:00:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.371+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fcc102e30 con 0x7f3fcc108300 2024-09-21T03:00:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.372+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f3fbc0107d0 con 0x7f3fcc108300 2024-09-21T03:00:04.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.373+0000 7f3fc9ffb640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 0x7f3fb80b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.373+0000 7f3fd0e52640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 0x7f3fb80b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.373+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f3fbc0e7ed0 con 0x7f3fcc108300 2024-09-21T03:00:04.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.373+0000 7f3fd0e52640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 0x7f3fb80b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40058c0 tx=0x7f3fb4005850 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.377+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f3fbc0ae600 con 0x7f3fcc108300 2024-09-21T03:00:04.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:04 smithi016 ceph-mon[31253]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T03:00:04.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3315801341' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:04.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.567+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3fcc02e640 con 0x7f3fcc108300 2024-09-21T03:00:04.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.567+0000 7f3fc9ffb640 1 -- 172.21.15.16:0/1914574337 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v79) ==== 93+0+10 (secure 0 0 0) 0x7f3fbc0b29e0 con 0x7f3fcc108300 2024-09-21T03:00:04.569 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-21T03:00:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 msgr2=0x7f3fb80b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 0x7f3fb80b8a70 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f3fb40058c0 tx=0x7f3fb4005850 comp rx=0 tx=0).stop 2024-09-21T03:00:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 msgr2=0x7f3fcc19a910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f3fbc00ebc0 tx=0x7f3fbc00c6a0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 shutdown_connections 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3fb80b6560 0x7f3fb80b8a70 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3fcc108300 0x7f3fcc19a910 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3fcc108c50 0x7f3fcc19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 --2- 172.21.15.16:0/1914574337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3fcc102a20 0x7f3fcc19a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 >> 172.21.15.16:0/1914574337 conn(0x7f3fcc079630 msgr2=0x7f3fcc104270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.569+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 shutdown_connections 2024-09-21T03:00:04.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.570+0000 7f3fd30b6640 1 -- 172.21.15.16:0/1914574337 wait complete. 2024-09-21T03:00:04.582 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:00:04.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:04 smithi018 ceph-mon[34587]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T03:00:04.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3315801341' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 -- 172.21.15.16:0/3224665396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c1082c0 msgr2=0x7f685c1086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 --2- 172.21.15.16:0/3224665396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c1082c0 0x7f685c1086d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f6844009900 tx=0x7f684402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 -- 172.21.15.16:0/3224665396 shutdown_connections 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 --2- 172.21.15.16:0/3224665396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c1082c0 0x7f685c1086d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 --2- 172.21.15.16:0/3224665396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c108c10 0x7f685c10f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 --2- 172.21.15.16:0/3224665396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c102990 0x7f685c102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 -- 172.21.15.16:0/3224665396 >> 172.21.15.16:0/3224665396 conn(0x7f685c0fe3b0 msgr2=0x7f685c100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 -- 172.21.15.16:0/3224665396 shutdown_connections 2024-09-21T03:00:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.697+0000 7f686224d640 1 -- 172.21.15.16:0/3224665396 wait complete. 2024-09-21T03:00:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 Processor -- start 2024-09-21T03:00:04.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 -- start start 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c102990 0x7f685c19a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c1082c0 0x7f685c19aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f685c112920 con 0x7f685c108c10 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f685c1127a0 con 0x7f685c102990 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f686224d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f685c112aa0 con 0x7f685c1082c0 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f685bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f685bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49556/0 (socket says 172.21.15.16:49556) 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f685bfff640 1 -- 172.21.15.16:0/2157241317 learned_addr learned my addr 172.21.15.16:0/2157241317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.698+0000 7f6852ffd640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c1082c0 0x7f685c19aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685b7fe640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c102990 0x7f685c19a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 -- 172.21.15.16:0/2157241317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c1082c0 msgr2=0x7f685c19aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c1082c0 0x7f685c19aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 -- 172.21.15.16:0/2157241317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c102990 msgr2=0x7f685c19a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c102990 0x7f685c19a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 -- 172.21.15.16:0/2157241317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f685c1a03c0 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f685bfff640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f685400eb10 tx=0x7f685400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854004b20 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6854004cc0 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6854010670 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f685c1a06b0 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.699+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f685c1a6e40 con 0x7f685c108c10 2024-09-21T03:00:04.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.700+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f685c102da0 con 0x7f685c108c10 2024-09-21T03:00:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.702+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f6854005000 con 0x7f685c108c10 2024-09-21T03:00:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.703+0000 7f68597fa640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 0x7f68480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.703+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f68540e7bd0 con 0x7f685c108c10 2024-09-21T03:00:04.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.703+0000 7f685b7fe640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 0x7f68480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:04.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.703+0000 7f685b7fe640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 0x7f68480b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f684c005fd0 tx=0x7f684c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:04.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.705+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6854056710 con 0x7f685c108c10 2024-09-21T03:00:04.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:04 smithi123 ceph-mon[34645]: pgmap v312: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 572 B/s rd, 0 B/s wr, 1 op/s 2024-09-21T03:00:04.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3315801341' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:04.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.896+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f685c10ba40 con 0x7f685c108c10 2024-09-21T03:00:04.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.896+0000 7f68597fa640 1 -- 172.21.15.16:0/2157241317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v79) ==== 104+0+11 (secure 0 0 0) 0x7f68540ae360 con 0x7f685c108c10 2024-09-21T03:00:04.898 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 msgr2=0x7f68480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 0x7f68480b8a70 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f684c005fd0 tx=0x7f684c00b390 comp rx=0 tx=0).stop 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 msgr2=0x7f685c19fcc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f685400eb10 tx=0x7f685400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 shutdown_connections 2024-09-21T03:00:04.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68480b6560 0x7f68480b8a70 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f685c108c10 0x7f685c19fcc0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685c1082c0 0x7f685c19aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 --2- 172.21.15.16:0/2157241317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f685c102990 0x7f685c19a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 >> 172.21.15.16:0/2157241317 conn(0x7f685c0fe3b0 msgr2=0x7f685c06aba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 shutdown_connections 2024-09-21T03:00:04.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:04.899+0000 7f686224d640 1 -- 172.21.15.16:0/2157241317 wait complete. 2024-09-21T03:00:04.912 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:00:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 -- 172.21.15.16:0/3165325678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65741082d0 msgr2=0x7f65741086e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 --2- 172.21.15.16:0/3165325678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65741082d0 0x7f65741086e0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f6564009900 tx=0x7f656402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 -- 172.21.15.16:0/3165325678 shutdown_connections 2024-09-21T03:00:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 --2- 172.21.15.16:0/3165325678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f65741082d0 0x7f65741086e0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 --2- 172.21.15.16:0/3165325678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6574108c20 0x7f657410f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 --2- 172.21.15.16:0/3165325678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f6574102db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 -- 172.21.15.16:0/3165325678 >> 172.21.15.16:0/3165325678 conn(0x7f65740fe3c0 msgr2=0x7f6574100810 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 -- 172.21.15.16:0/3165325678 shutdown_connections 2024-09-21T03:00:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.027+0000 7f657b440640 1 -- 172.21.15.16:0/3165325678 wait complete. 2024-09-21T03:00:05.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 Processor -- start 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 -- start start 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f657419a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65741082d0 0x7f657419a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6574113a00 con 0x7f6574108c20 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6574113880 con 0x7f65741029a0 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f657b440640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6574113b80 con 0x7f65741082d0 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f65799dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f65799dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49570/0 (socket says 172.21.15.16:49570) 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.028+0000 7f65799dd640 1 -- 172.21.15.16:0/3147660113 learned_addr learned my addr 172.21.15.16:0/3147660113 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65791dc640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f657419a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65789db640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65741082d0 0x7f657419a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 -- 172.21.15.16:0/3147660113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65741082d0 msgr2=0x7f657419a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65741082d0 0x7f657419a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 -- 172.21.15.16:0/3147660113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 msgr2=0x7f657419a420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f657419a420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 -- 172.21.15.16:0/3147660113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65741a0350 con 0x7f6574108c20 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65791dc640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f657419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f65799dd640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c017070 con 0x7f6574108c20 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c003e20 con 0x7f6574108c20 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65741a0640 con 0x7f6574108c20 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c010460 con 0x7f6574108c20 2024-09-21T03:00:05.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.029+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65741a6dd0 con 0x7f6574108c20 2024-09-21T03:00:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.030+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6574103c10 con 0x7f6574108c20 2024-09-21T03:00:05.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.032+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f656c003970 con 0x7f6574108c20 2024-09-21T03:00:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.033+0000 7f656a7fc640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 0x7f65600b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.033+0000 7f65791dc640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 0x7f65600b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.033+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f656c0e7280 con 0x7f6574108c20 2024-09-21T03:00:05.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.034+0000 7f65791dc640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 0x7f65600b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6570009c20 tx=0x7f6570005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.037+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f656c0ada10 con 0x7f6574108c20 2024-09-21T03:00:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.224+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f657410a9f0 con 0x7f6574108c20 2024-09-21T03:00:05.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.225+0000 7f656a7fc640 1 -- 172.21.15.16:0/3147660113 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v79) ==== 100+0+11 (secure 0 0 0) 0x7f656c0b1df0 con 0x7f6574108c20 2024-09-21T03:00:05.227 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 msgr2=0x7f65600b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 0x7f65600b8a70 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f6570009c20 tx=0x7f6570005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 msgr2=0x7f657419fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f656c009fd0 tx=0x7f656c00b260 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 shutdown_connections 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65600b6560 0x7f65600b8a70 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6574108c20 0x7f657419fc50 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65741082d0 0x7f657419a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 --2- 172.21.15.16:0/3147660113 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f65741029a0 0x7f657419a420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 >> 172.21.15.16:0/3147660113 conn(0x7f65740fe3c0 msgr2=0x7f65741007e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 shutdown_connections 2024-09-21T03:00:05.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.227+0000 7f657b440640 1 -- 172.21.15.16:0/3147660113 wait complete. 2024-09-21T03:00:05.240 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-09-21T03:00:05.240 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:05.499 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:05.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1914574337' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2157241317' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3147660113' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1914574337' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2157241317' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3147660113' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1914574337' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2157241317' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3147660113' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 -- 172.21.15.16:0/2795833603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c102ae0 msgr2=0x7f804c102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 --2- 172.21.15.16:0/2795833603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c102ae0 0x7f804c102ef0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f8034009900 tx=0x7f803402a3f0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 -- 172.21.15.16:0/2795833603 shutdown_connections 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 --2- 172.21.15.16:0/2795833603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 0x7f804c10f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 --2- 172.21.15.16:0/2795833603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c1083d0 0x7f804c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 --2- 172.21.15.16:0/2795833603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c102ae0 0x7f804c102ef0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 -- 172.21.15.16:0/2795833603 >> 172.21.15.16:0/2795833603 conn(0x7f804c079760 msgr2=0x7f804c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 -- 172.21.15.16:0/2795833603 shutdown_connections 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.986+0000 7f8053181640 1 -- 172.21.15.16:0/2795833603 wait complete. 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 Processor -- start 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 -- start start 2024-09-21T03:00:05.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c102ae0 0x7f804c104510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 0x7f804c104f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f804c112990 con 0x7f804c1083d0 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f804c112810 con 0x7f804c108d20 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f8053181640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f804c112b10 con 0x7f804c102ae0 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.987+0000 7f804bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49574/0 (socket says 172.21.15.16:49574) 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 -- 172.21.15.16:0/1809174217 learned_addr learned my addr 172.21.15.16:0/1809174217 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f8050f1d640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c102ae0 0x7f804c104510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f805171e640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 0x7f804c104f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 -- 172.21.15.16:0/1809174217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c102ae0 msgr2=0x7f804c104510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c102ae0 0x7f804c104510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 -- 172.21.15.16:0/1809174217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 msgr2=0x7f804c104f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:05.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 0x7f804c104f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 -- 172.21.15.16:0/1809174217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f804c1a86a0 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f804bfff640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c970 tx=0x7f803c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c014070 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f804c1a8990 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f803c004440 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.988+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f803c003e50 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.989+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f804c1a8f20 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.990+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f804c103d10 con 0x7f804c1083d0 2024-09-21T03:00:05.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.991+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f803c017070 con 0x7f804c1083d0 2024-09-21T03:00:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.992+0000 7f8049ffb640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 0x7f80380b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.992+0000 7f8050f1d640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 0x7f80380b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.992+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f803c0e7660 con 0x7f804c1083d0 2024-09-21T03:00:05.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.992+0000 7f8050f1d640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 0x7f80380b8c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8034000c00 tx=0x7f8034031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:05.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:05.995+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f803c01e020 con 0x7f804c1083d0 2024-09-21T03:00:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.219+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f804c102ef0 con 0x7f804c1083d0 2024-09-21T03:00:06.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.220+0000 7f8049ffb640 1 -- 172.21.15.16:0/1809174217 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f804c102ef0 con 0x7f804c1083d0 2024-09-21T03:00:06.221 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:06.222 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:00:06.222 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:00:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 msgr2=0x7f80380b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 0x7f80380b8c10 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f8034000c00 tx=0x7f8034031400 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 msgr2=0x7f804c104a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f803c00c970 tx=0x7f803c00ce30 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 shutdown_connections 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f80380b6700 0x7f80380b8c10 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f804c1083d0 0x7f804c104a50 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f804c108d20 0x7f804c104f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 --2- 172.21.15.16:0/1809174217 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f804c102ae0 0x7f804c104510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 >> 172.21.15.16:0/1809174217 conn(0x7f804c079760 msgr2=0x7f804c106880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 shutdown_connections 2024-09-21T03:00:06.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:06.223+0000 7f8053181640 1 -- 172.21.15.16:0/1809174217 wait complete. 2024-09-21T03:00:06.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:06 smithi016 ceph-mon[31253]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:00:06.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:06 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1809174217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:06.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:06 smithi018 ceph-mon[34587]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:00:06.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:06 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1809174217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:06.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:06 smithi123 ceph-mon[34645]: pgmap v313: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:00:06.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:06 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1809174217' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:06.913 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-09-21T03:00:06.913 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-09-21T03:00:06.913 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 -- 172.21.15.16:0/2020773893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 msgr2=0x7f701c103ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 --2- 172.21.15.16:0/2020773893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c103ef0 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f7018009900 tx=0x7f701802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 -- 172.21.15.16:0/2020773893 shutdown_connections 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 --2- 172.21.15.16:0/2020773893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c110590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 --2- 172.21.15.16:0/2020773893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 0x7f701c109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 --2- 172.21.15.16:0/2020773893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c103ef0 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 -- 172.21.15.16:0/2020773893 >> 172.21.15.16:0/2020773893 conn(0x7f701c0ff500 msgr2=0x7f701c101950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 -- 172.21.15.16:0/2020773893 shutdown_connections 2024-09-21T03:00:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.032+0000 7f702420e640 1 -- 172.21.15.16:0/2020773893 wait complete. 2024-09-21T03:00:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 Processor -- start 2024-09-21T03:00:07.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 -- start start 2024-09-21T03:00:07.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 0x7f701c19a9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c19fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f701c113a20 con 0x7f701c103ae0 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f701c1138a0 con 0x7f701c109d60 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f702420e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f701c113ba0 con 0x7f701c109410 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f7021faa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f7021faa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49592/0 (socket says 172.21.15.16:49592) 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f7021faa640 1 -- 172.21.15.16:0/2559624540 learned_addr learned my addr 172.21.15.16:0/2559624540 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.033+0000 7f70217a9640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 0x7f701c19a9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f70227ab640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c19fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 -- 172.21.15.16:0/2559624540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 msgr2=0x7f701c19a9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 0x7f701c19a9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 -- 172.21.15.16:0/2559624540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 msgr2=0x7f701c19fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c19fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 -- 172.21.15.16:0/2559624540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f701c1a0350 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f70227ab640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7021faa640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7018005940 tx=0x7f7018004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701803f030 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7018003aa0 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7018035070 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f701c1a05e0 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.034+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f701c1a6dd0 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.035+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f701c103ef0 con 0x7f701c103ae0 2024-09-21T03:00:07.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.037+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f7018042030 con 0x7f701c103ae0 2024-09-21T03:00:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.038+0000 7f7012ffd640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 0x7f70080b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.038+0000 7f70217a9640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 0x7f70080b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.038+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f7018107e80 con 0x7f701c103ae0 2024-09-21T03:00:07.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.039+0000 7f70217a9640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 0x7f70080b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f701c108f20 tx=0x7f700c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.042+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f70180ce610 con 0x7f701c103ae0 2024-09-21T03:00:07.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.226+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f701c063800 con 0x7f70080b6560 2024-09-21T03:00:07.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.230+0000 7f7012ffd640 1 -- 172.21.15.16:0/2559624540 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258731 (secure 0 0 0) 0x7f701c063800 con 0x7f70080b6560 2024-09-21T03:00:07.234 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:07.235 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 msgr2=0x7f70080b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 0x7f70080b8a70 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f701c108f20 tx=0x7f700c005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 msgr2=0x7f701c19a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7018005940 tx=0x7f7018004270 comp rx=0 tx=0).stop 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 shutdown_connections 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70080b6560 0x7f70080b8a70 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f701c109d60 0x7f701c19fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f701c109410 0x7f701c19a9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 --2- 172.21.15.16:0/2559624540 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f701c103ae0 0x7f701c19a420 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 >> 172.21.15.16:0/2559624540 conn(0x7f701c0ff500 msgr2=0x7f701c101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 shutdown_connections 2024-09-21T03:00:07.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.234+0000 7f702420e640 1 -- 172.21.15.16:0/2559624540 wait complete. 2024-09-21T03:00:07.247 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T03:00:07.173170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390668,"kb_used_data":12228,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551156,"statfs":{"total":1151940427776,"available":1151540383744,"internally_reserved":0,"allocated":12521472,"data_stored":7136306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.933954"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247799+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:59:55.247799+0000","last_peered":"2024-09-21T02:59:55.247799+0000","last_clean":"2024-09-21T02:59:55.247799+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:59:55.247799+0000","last_undegraded":"2024-09-21T02:59:55.247799+0000","last_fullsized":"2024-09-21T02:59:55.247799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796617+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:59:55.796617+0000","last_peered":"2024-09-21T02:59:55.796617+0000","last_clean":"2024-09-21T02:59:55.796617+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:59:55.796617+0000","last_undegraded":"2024-09-21T02:59:55.796617+0000","last_fullsized":"2024-09-21T02:59:55.796617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796611+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:59:55.796611+0000","last_peered":"2024-09-21T02:59:55.796611+0000","last_clean":"2024-09-21T02:59:55.796611+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:59:55.796611+0000","last_undegraded":"2024-09-21T02:59:55.796611+0000","last_fullsized":"2024-09-21T02:59:55.796611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242650+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:59:55.242650+0000","last_peered":"2024-09-21T02:59:55.242650+0000","last_clean":"2024-09-21T02:59:55.242650+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:59:55.242650+0000","last_undegraded":"2024-09-21T02:59:55.242650+0000","last_fullsized":"2024-09-21T02:59:55.242650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769885+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:59:55.769885+0000","last_peered":"2024-09-21T02:59:55.769885+0000","last_clean":"2024-09-21T02:59:55.769885+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:59:55.769885+0000","last_undegraded":"2024-09-21T02:59:55.769885+0000","last_fullsized":"2024-09-21T02:59:55.769885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769866+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:59:55.769866+0000","last_peered":"2024-09-21T02:59:55.769866+0000","last_clean":"2024-09-21T02:59:55.769866+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:59:55.769866+0000","last_undegraded":"2024-09-21T02:59:55.769866+0000","last_fullsized":"2024-09-21T02:59:55.769866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466197+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:59:55.466197+0000","last_peered":"2024-09-21T02:59:55.466197+0000","last_clean":"2024-09-21T02:59:55.466197+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:59:55.466197+0000","last_undegraded":"2024-09-21T02:59:55.466197+0000","last_fullsized":"2024-09-21T02:59:55.466197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769683+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:59:55.769683+0000","last_peered":"2024-09-21T02:59:55.769683+0000","last_clean":"2024-09-21T02:59:55.769683+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:59:55.769683+0000","last_undegraded":"2024-09-21T02:59:55.769683+0000","last_fullsized":"2024-09-21T02:59:55.769683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769663+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:59:55.769663+0000","last_peered":"2024-09-21T02:59:55.769663+0000","last_clean":"2024-09-21T02:59:55.769663+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:59:55.769663+0000","last_undegraded":"2024-09-21T02:59:55.769663+0000","last_fullsized":"2024-09-21T02:59:55.769663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769926+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:59:55.769926+0000","last_peered":"2024-09-21T02:59:55.769926+0000","last_clean":"2024-09-21T02:59:55.769926+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:59:55.769926+0000","last_undegraded":"2024-09-21T02:59:55.769926+0000","last_fullsized":"2024-09-21T02:59:55.769926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466653+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:59:55.466653+0000","last_peered":"2024-09-21T02:59:55.466653+0000","last_clean":"2024-09-21T02:59:55.466653+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:59:55.466653+0000","last_undegraded":"2024-09-21T02:59:55.466653+0000","last_fullsized":"2024-09-21T02:59:55.466653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242827+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:59:55.242827+0000","last_peered":"2024-09-21T02:59:55.242827+0000","last_clean":"2024-09-21T02:59:55.242827+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:59:55.242827+0000","last_undegraded":"2024-09-21T02:59:55.242827+0000","last_fullsized":"2024-09-21T02:59:55.242827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241546+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:59:55.241546+0000","last_peered":"2024-09-21T02:59:55.241546+0000","last_clean":"2024-09-21T02:59:55.241546+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:59:55.241546+0000","last_undegraded":"2024-09-21T02:59:55.241546+0000","last_fullsized":"2024-09-21T02:59:55.241546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796764+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:59:55.796764+0000","last_peered":"2024-09-21T02:59:55.796764+0000","last_clean":"2024-09-21T02:59:55.796764+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:59:55.796764+0000","last_undegraded":"2024-09-21T02:59:55.796764+0000","last_fullsized":"2024-09-21T02:59:55.796764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796581+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:59:55.796581+0000","last_peered":"2024-09-21T02:59:55.796581+0000","last_clean":"2024-09-21T02:59:55.796581+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:59:55.796581+0000","last_undegraded":"2024-09-21T02:59:55.796581+0000","last_fullsized":"2024-09-21T02:59:55.796581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770150+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:59:55.770150+0000","last_peered":"2024-09-21T02:59:55.770150+0000","last_clean":"2024-09-21T02:59:55.770150+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:59:55.770150+0000","last_undegraded":"2024-09-21T02:59:55.770150+0000","last_fullsized":"2024-09-21T02:59:55.770150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247833+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:59:55.247833+0000","last_peered":"2024-09-21T02:59:55.247833+0000","last_clean":"2024-09-21T02:59:55.247833+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:59:55.247833+0000","last_undegraded":"2024-09-21T02:59:55.247833+0000","last_fullsized":"2024-09-21T02:59:55.247833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770586+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:59:55.770586+0000","last_peered":"2024-09-21T02:59:55.770586+0000","last_clean":"2024-09-21T02:59:55.770586+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:59:55.770586+0000","last_undegraded":"2024-09-21T02:59:55.770586+0000","last_fullsized":"2024-09-21T02:59:55.770586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770832+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:59:55.770832+0000","last_peered":"2024-09-21T02:59:55.770832+0000","last_clean":"2024-09-21T02:59:55.770832+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:59:55.770832+0000","last_undegraded":"2024-09-21T02:59:55.770832+0000","last_fullsized":"2024-09-21T02:59:55.770832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247315+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:59:55.247315+0000","last_peered":"2024-09-21T02:59:55.247315+0000","last_clean":"2024-09-21T02:59:55.247315+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:59:55.247315+0000","last_undegraded":"2024-09-21T02:59:55.247315+0000","last_fullsized":"2024-09-21T02:59:55.247315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241585+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:59:55.241585+0000","last_peered":"2024-09-21T02:59:55.241585+0000","last_clean":"2024-09-21T02:59:55.241585+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:59:55.241585+0000","last_undegraded":"2024-09-21T02:59:55.241585+0000","last_fullsized":"2024-09-21T02:59:55.241585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770370+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:59:55.770370+0000","last_peered":"2024-09-21T02:59:55.770370+0000","last_clean":"2024-09-21T02:59:55.770370+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:59:55.770370+0000","last_undegraded":"2024-09-21T02:59:55.770370+0000","last_fullsized":"2024-09-21T02:59:55.770370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770410+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:59:55.770410+0000","last_peered":"2024-09-21T02:59:55.770410+0000","last_clean":"2024-09-21T02:59:55.770410+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:59:55.770410+0000","last_undegraded":"2024-09-21T02:59:55.770410+0000","last_fullsized":"2024-09-21T02:59:55.770410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245184+0000","last_change":"2024-09-21T02:58:54.745262+0000","last_active":"2024-09-21T02:59:55.245184+0000","last_peered":"2024-09-21T02:59:55.245184+0000","last_clean":"2024-09-21T02:59:55.245184+0000","last_became_active":"2024-09-21T02:58:54.744809+0000","last_became_peered":"2024-09-21T02:58:54.744809+0000","last_unstale":"2024-09-21T02:59:55.245184+0000","last_undegraded":"2024-09-21T02:59:55.245184+0000","last_fullsized":"2024-09-21T02:59:55.245184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:32.627252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770274+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:59:55.770274+0000","last_peered":"2024-09-21T02:59:55.770274+0000","last_clean":"2024-09-21T02:59:55.770274+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:59:55.770274+0000","last_undegraded":"2024-09-21T02:59:55.770274+0000","last_fullsized":"2024-09-21T02:59:55.770274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770897+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:59:55.770897+0000","last_peered":"2024-09-21T02:59:55.770897+0000","last_clean":"2024-09-21T02:59:55.770897+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:59:55.770897+0000","last_undegraded":"2024-09-21T02:59:55.770897+0000","last_fullsized":"2024-09-21T02:59:55.770897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247598+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:59:55.247598+0000","last_peered":"2024-09-21T02:59:55.247598+0000","last_clean":"2024-09-21T02:59:55.247598+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:59:55.247598+0000","last_undegraded":"2024-09-21T02:59:55.247598+0000","last_fullsized":"2024-09-21T02:59:55.247598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241323+0000","last_change":"2024-09-21T02:58:54.744120+0000","last_active":"2024-09-21T02:59:55.241323+0000","last_peered":"2024-09-21T02:59:55.241323+0000","last_clean":"2024-09-21T02:59:55.241323+0000","last_became_active":"2024-09-21T02:58:54.743669+0000","last_became_peered":"2024-09-21T02:58:54.743669+0000","last_unstale":"2024-09-21T02:59:55.241323+0000","last_undegraded":"2024-09-21T02:59:55.241323+0000","last_fullsized":"2024-09-21T02:59:55.241323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:16.956302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796609+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:59:55.796609+0000","last_peered":"2024-09-21T02:59:55.796609+0000","last_clean":"2024-09-21T02:59:55.796609+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:59:55.796609+0000","last_undegraded":"2024-09-21T02:59:55.796609+0000","last_fullsized":"2024-09-21T02:59:55.796609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770512+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:59:55.770512+0000","last_peered":"2024-09-21T02:59:55.770512+0000","last_clean":"2024-09-21T02:59:55.770512+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:59:55.770512+0000","last_undegraded":"2024-09-21T02:59:55.770512+0000","last_fullsized":"2024-09-21T02:59:55.770512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770484+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:59:55.770484+0000","last_peered":"2024-09-21T02:59:55.770484+0000","last_clean":"2024-09-21T02:59:55.770484+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:59:55.770484+0000","last_undegraded":"2024-09-21T02:59:55.770484+0000","last_fullsized":"2024-09-21T02:59:55.770484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247962+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:59:55.247962+0000","last_peered":"2024-09-21T02:59:55.247962+0000","last_clean":"2024-09-21T02:59:55.247962+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:59:55.247962+0000","last_undegraded":"2024-09-21T02:59:55.247962+0000","last_fullsized":"2024-09-21T02:59:55.247962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769880+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:59:55.769880+0000","last_peered":"2024-09-21T02:59:55.769880+0000","last_clean":"2024-09-21T02:59:55.769880+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:59:55.769880+0000","last_undegraded":"2024-09-21T02:59:55.769880+0000","last_fullsized":"2024-09-21T02:59:55.769880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245399+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:59:55.245399+0000","last_peered":"2024-09-21T02:59:55.245399+0000","last_clean":"2024-09-21T02:59:55.245399+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:59:55.245399+0000","last_undegraded":"2024-09-21T02:59:55.245399+0000","last_fullsized":"2024-09-21T02:59:55.245399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245319+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:59:55.245319+0000","last_peered":"2024-09-21T02:59:55.245319+0000","last_clean":"2024-09-21T02:59:55.245319+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:59:55.245319+0000","last_undegraded":"2024-09-21T02:59:55.245319+0000","last_fullsized":"2024-09-21T02:59:55.245319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770839+0000","last_change":"2024-09-21T02:58:54.912079+0000","last_active":"2024-09-21T02:59:55.770839+0000","last_peered":"2024-09-21T02:59:55.770839+0000","last_clean":"2024-09-21T02:59:55.770839+0000","last_became_active":"2024-09-21T02:58:54.911451+0000","last_became_peered":"2024-09-21T02:58:54.911451+0000","last_unstale":"2024-09-21T02:59:55.770839+0000","last_undegraded":"2024-09-21T02:59:55.770839+0000","last_fullsized":"2024-09-21T02:59:55.770839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:02.692841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240875+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:59:55.240875+0000","last_peered":"2024-09-21T02:59:55.240875+0000","last_clean":"2024-09-21T02:59:55.240875+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:59:55.240875+0000","last_undegraded":"2024-09-21T02:59:55.240875+0000","last_fullsized":"2024-09-21T02:59:55.240875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769478+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:59:55.769478+0000","last_peered":"2024-09-21T02:59:55.769478+0000","last_clean":"2024-09-21T02:59:55.769478+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:59:55.769478+0000","last_undegraded":"2024-09-21T02:59:55.769478+0000","last_fullsized":"2024-09-21T02:59:55.769478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.773488+0000","last_change":"2024-09-21T02:59:55.772969+0000","last_active":"2024-09-21T02:59:55.773488+0000","last_peered":"2024-09-21T02:59:55.773488+0000","last_clean":"2024-09-21T02:59:55.773488+0000","last_became_active":"2024-09-21T02:59:55.772407+0000","last_became_peered":"2024-09-21T02:59:55.772407+0000","last_unstale":"2024-09-21T02:59:55.773488+0000","last_undegraded":"2024-09-21T02:59:55.773488+0000","last_fullsized":"2024-09-21T02:59:55.773488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:18.002558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769606+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:59:55.769606+0000","last_peered":"2024-09-21T02:59:55.769606+0000","last_clean":"2024-09-21T02:59:55.769606+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:59:55.769606+0000","last_undegraded":"2024-09-21T02:59:55.769606+0000","last_fullsized":"2024-09-21T02:59:55.769606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771476+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:59:55.771476+0000","last_peered":"2024-09-21T02:59:55.771476+0000","last_clean":"2024-09-21T02:59:55.771476+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:59:55.771476+0000","last_undegraded":"2024-09-21T02:59:55.771476+0000","last_fullsized":"2024-09-21T02:59:55.771476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771071+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:59:55.771071+0000","last_peered":"2024-09-21T02:59:55.771071+0000","last_clean":"2024-09-21T02:59:55.771071+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:59:55.771071+0000","last_undegraded":"2024-09-21T02:59:55.771071+0000","last_fullsized":"2024-09-21T02:59:55.771071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466729+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:59:55.466729+0000","last_peered":"2024-09-21T02:59:55.466729+0000","last_clean":"2024-09-21T02:59:55.466729+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:59:55.466729+0000","last_undegraded":"2024-09-21T02:59:55.466729+0000","last_fullsized":"2024-09-21T02:59:55.466729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797346+0000","last_change":"2024-09-21T02:58:54.745298+0000","last_active":"2024-09-21T02:59:55.797346+0000","last_peered":"2024-09-21T02:59:55.797346+0000","last_clean":"2024-09-21T02:59:55.797346+0000","last_became_active":"2024-09-21T02:58:54.744775+0000","last_became_peered":"2024-09-21T02:58:54.744775+0000","last_unstale":"2024-09-21T02:59:55.797346+0000","last_undegraded":"2024-09-21T02:59:55.797346+0000","last_fullsized":"2024-09-21T02:59:55.797346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:14.405920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245494+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:59:55.245494+0000","last_peered":"2024-09-21T02:59:55.245494+0000","last_clean":"2024-09-21T02:59:55.245494+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:59:55.245494+0000","last_undegraded":"2024-09-21T02:59:55.245494+0000","last_fullsized":"2024-09-21T02:59:55.245494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769650+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:59:55.769650+0000","last_peered":"2024-09-21T02:59:55.769650+0000","last_clean":"2024-09-21T02:59:55.769650+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:59:55.769650+0000","last_undegraded":"2024-09-21T02:59:55.769650+0000","last_fullsized":"2024-09-21T02:59:55.769650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":247,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:00:03.830470+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T03:00:03.830470+0000","last_peered":"2024-09-21T03:00:03.830470+0000","last_clean":"2024-09-21T03:00:03.830470+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T03:00:03.830470+0000","last_undegraded":"2024-09-21T03:00:03.830470+0000","last_fullsized":"2024-09-21T03:00:03.830470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":160,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769962+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:59:55.769962+0000","last_peered":"2024-09-21T02:59:55.769962+0000","last_clean":"2024-09-21T02:59:55.769962+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:59:55.769962+0000","last_undegraded":"2024-09-21T02:59:55.769962+0000","last_fullsized":"2024-09-21T02:59:55.769962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771792+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:59:55.771792+0000","last_peered":"2024-09-21T02:59:55.771792+0000","last_clean":"2024-09-21T02:59:55.771792+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:59:55.771792+0000","last_undegraded":"2024-09-21T02:59:55.771792+0000","last_fullsized":"2024-09-21T02:59:55.771792+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240960+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:59:55.240960+0000","last_peered":"2024-09-21T02:59:55.240960+0000","last_clean":"2024-09-21T02:59:55.240960+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:59:55.240960+0000","last_undegraded":"2024-09-21T02:59:55.240960+0000","last_fullsized":"2024-09-21T02:59:55.240960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245476+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:59:55.245476+0000","last_peered":"2024-09-21T02:59:55.245476+0000","last_clean":"2024-09-21T02:59:55.245476+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:59:55.245476+0000","last_undegraded":"2024-09-21T02:59:55.245476+0000","last_fullsized":"2024-09-21T02:59:55.245476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466508+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:59:55.466508+0000","last_peered":"2024-09-21T02:59:55.466508+0000","last_clean":"2024-09-21T02:59:55.466508+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:59:55.466508+0000","last_undegraded":"2024-09-21T02:59:55.466508+0000","last_fullsized":"2024-09-21T02:59:55.466508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242473+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:59:55.242473+0000","last_peered":"2024-09-21T02:59:55.242473+0000","last_clean":"2024-09-21T02:59:55.242473+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:59:55.242473+0000","last_undegraded":"2024-09-21T02:59:55.242473+0000","last_fullsized":"2024-09-21T02:59:55.242473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466307+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:59:55.466307+0000","last_peered":"2024-09-21T02:59:55.466307+0000","last_clean":"2024-09-21T02:59:55.466307+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:59:55.466307+0000","last_undegraded":"2024-09-21T02:59:55.466307+0000","last_fullsized":"2024-09-21T02:59:55.466307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770297+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:59:55.770297+0000","last_peered":"2024-09-21T02:59:55.770297+0000","last_clean":"2024-09-21T02:59:55.770297+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:59:55.770297+0000","last_undegraded":"2024-09-21T02:59:55.770297+0000","last_fullsized":"2024-09-21T02:59:55.770297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245247+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:59:55.245247+0000","last_peered":"2024-09-21T02:59:55.245247+0000","last_clean":"2024-09-21T02:59:55.245247+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:59:55.245247+0000","last_undegraded":"2024-09-21T02:59:55.245247+0000","last_fullsized":"2024-09-21T02:59:55.245247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771747+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:59:55.771747+0000","last_peered":"2024-09-21T02:59:55.771747+0000","last_clean":"2024-09-21T02:59:55.771747+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:59:55.771747+0000","last_undegraded":"2024-09-21T02:59:55.771747+0000","last_fullsized":"2024-09-21T02:59:55.771747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247441+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:59:55.247441+0000","last_peered":"2024-09-21T02:59:55.247441+0000","last_clean":"2024-09-21T02:59:55.247441+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:59:55.247441+0000","last_undegraded":"2024-09-21T02:59:55.247441+0000","last_fullsized":"2024-09-21T02:59:55.247441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241296+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:59:55.241296+0000","last_peered":"2024-09-21T02:59:55.241296+0000","last_clean":"2024-09-21T02:59:55.241296+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:59:55.241296+0000","last_undegraded":"2024-09-21T02:59:55.241296+0000","last_fullsized":"2024-09-21T02:59:55.241296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466773+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:59:55.466773+0000","last_peered":"2024-09-21T02:59:55.466773+0000","last_clean":"2024-09-21T02:59:55.466773+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:59:55.466773+0000","last_undegraded":"2024-09-21T02:59:55.466773+0000","last_fullsized":"2024-09-21T02:59:55.466773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245695+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:59:55.245695+0000","last_peered":"2024-09-21T02:59:55.245695+0000","last_clean":"2024-09-21T02:59:55.245695+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:59:55.245695+0000","last_undegraded":"2024-09-21T02:59:55.245695+0000","last_fullsized":"2024-09-21T02:59:55.245695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466573+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:59:55.466573+0000","last_peered":"2024-09-21T02:59:55.466573+0000","last_clean":"2024-09-21T02:59:55.466573+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:59:55.466573+0000","last_undegraded":"2024-09-21T02:59:55.466573+0000","last_fullsized":"2024-09-21T02:59:55.466573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.772036+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:59:55.772036+0000","last_peered":"2024-09-21T02:59:55.772036+0000","last_clean":"2024-09-21T02:59:55.772036+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:59:55.772036+0000","last_undegraded":"2024-09-21T02:59:55.772036+0000","last_fullsized":"2024-09-21T02:59:55.772036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466818+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:59:55.466818+0000","last_peered":"2024-09-21T02:59:55.466818+0000","last_clean":"2024-09-21T02:59:55.466818+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:59:55.466818+0000","last_undegraded":"2024-09-21T02:59:55.466818+0000","last_fullsized":"2024-09-21T02:59:55.466818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466691+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:59:55.466691+0000","last_peered":"2024-09-21T02:59:55.466691+0000","last_clean":"2024-09-21T02:59:55.466691+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:59:55.466691+0000","last_undegraded":"2024-09-21T02:59:55.466691+0000","last_fullsized":"2024-09-21T02:59:55.466691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242571+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:59:55.242571+0000","last_peered":"2024-09-21T02:59:55.242571+0000","last_clean":"2024-09-21T02:59:55.242571+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:59:55.242571+0000","last_undegraded":"2024-09-21T02:59:55.242571+0000","last_fullsized":"2024-09-21T02:59:55.242571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769526+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:59:55.769526+0000","last_peered":"2024-09-21T02:59:55.769526+0000","last_clean":"2024-09-21T02:59:55.769526+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:59:55.769526+0000","last_undegraded":"2024-09-21T02:59:55.769526+0000","last_fullsized":"2024-09-21T02:59:55.769526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466591+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:59:55.466591+0000","last_peered":"2024-09-21T02:59:55.466591+0000","last_clean":"2024-09-21T02:59:55.466591+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:59:55.466591+0000","last_undegraded":"2024-09-21T02:59:55.466591+0000","last_fullsized":"2024-09-21T02:59:55.466591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240903+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:59:55.240903+0000","last_peered":"2024-09-21T02:59:55.240903+0000","last_clean":"2024-09-21T02:59:55.240903+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:59:55.240903+0000","last_undegraded":"2024-09-21T02:59:55.240903+0000","last_fullsized":"2024-09-21T02:59:55.240903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241097+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:59:55.241097+0000","last_peered":"2024-09-21T02:59:55.241097+0000","last_clean":"2024-09-21T02:59:55.241097+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:59:55.241097+0000","last_undegraded":"2024-09-21T02:59:55.241097+0000","last_fullsized":"2024-09-21T02:59:55.241097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770245+0000","last_change":"2024-09-21T02:58:54.909889+0000","last_active":"2024-09-21T02:59:55.770245+0000","last_peered":"2024-09-21T02:59:55.770245+0000","last_clean":"2024-09-21T02:59:55.770245+0000","last_became_active":"2024-09-21T02:58:54.909607+0000","last_became_peered":"2024-09-21T02:58:54.909607+0000","last_unstale":"2024-09-21T02:59:55.770245+0000","last_undegraded":"2024-09-21T02:59:55.770245+0000","last_fullsized":"2024-09-21T02:59:55.770245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466532+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:59:55.466532+0000","last_peered":"2024-09-21T02:59:55.466532+0000","last_clean":"2024-09-21T02:59:55.466532+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:59:55.466532+0000","last_undegraded":"2024-09-21T02:59:55.466532+0000","last_fullsized":"2024-09-21T02:59:55.466532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769978+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:59:55.769978+0000","last_peered":"2024-09-21T02:59:55.769978+0000","last_clean":"2024-09-21T02:59:55.769978+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:59:55.769978+0000","last_undegraded":"2024-09-21T02:59:55.769978+0000","last_fullsized":"2024-09-21T02:59:55.769978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770430+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:59:55.770430+0000","last_peered":"2024-09-21T02:59:55.770430+0000","last_clean":"2024-09-21T02:59:55.770430+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:59:55.770430+0000","last_undegraded":"2024-09-21T02:59:55.770430+0000","last_fullsized":"2024-09-21T02:59:55.770430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769861+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:59:55.769861+0000","last_peered":"2024-09-21T02:59:55.769861+0000","last_clean":"2024-09-21T02:59:55.769861+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:59:55.769861+0000","last_undegraded":"2024-09-21T02:59:55.769861+0000","last_fullsized":"2024-09-21T02:59:55.769861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771522+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:59:55.771522+0000","last_peered":"2024-09-21T02:59:55.771522+0000","last_clean":"2024-09-21T02:59:55.771522+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:59:55.771522+0000","last_undegraded":"2024-09-21T02:59:55.771522+0000","last_fullsized":"2024-09-21T02:59:55.771522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241371+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:59:55.241371+0000","last_peered":"2024-09-21T02:59:55.241371+0000","last_clean":"2024-09-21T02:59:55.241371+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:59:55.241371+0000","last_undegraded":"2024-09-21T02:59:55.241371+0000","last_fullsized":"2024-09-21T02:59:55.241371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770670+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:59:55.770670+0000","last_peered":"2024-09-21T02:59:55.770670+0000","last_clean":"2024-09-21T02:59:55.770670+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:59:55.770670+0000","last_undegraded":"2024-09-21T02:59:55.770670+0000","last_fullsized":"2024-09-21T02:59:55.770670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770035+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:59:55.770035+0000","last_peered":"2024-09-21T02:59:55.770035+0000","last_clean":"2024-09-21T02:59:55.770035+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:59:55.770035+0000","last_undegraded":"2024-09-21T02:59:55.770035+0000","last_fullsized":"2024-09-21T02:59:55.770035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242807+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:59:55.242807+0000","last_peered":"2024-09-21T02:59:55.242807+0000","last_clean":"2024-09-21T02:59:55.242807+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:59:55.242807+0000","last_undegraded":"2024-09-21T02:59:55.242807+0000","last_fullsized":"2024-09-21T02:59:55.242807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797033+0000","last_change":"2024-09-21T02:58:54.743546+0000","last_active":"2024-09-21T02:59:55.797033+0000","last_peered":"2024-09-21T02:59:55.797033+0000","last_clean":"2024-09-21T02:59:55.797033+0000","last_became_active":"2024-09-21T02:58:54.743107+0000","last_became_peered":"2024-09-21T02:58:54.743107+0000","last_unstale":"2024-09-21T02:59:55.797033+0000","last_undegraded":"2024-09-21T02:59:55.797033+0000","last_fullsized":"2024-09-21T02:59:55.797033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.458721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770566+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:59:55.770566+0000","last_peered":"2024-09-21T02:59:55.770566+0000","last_clean":"2024-09-21T02:59:55.770566+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:59:55.770566+0000","last_undegraded":"2024-09-21T02:59:55.770566+0000","last_fullsized":"2024-09-21T02:59:55.770566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769867+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:59:55.769867+0000","last_peered":"2024-09-21T02:59:55.769867+0000","last_clean":"2024-09-21T02:59:55.769867+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:59:55.769867+0000","last_undegraded":"2024-09-21T02:59:55.769867+0000","last_fullsized":"2024-09-21T02:59:55.769867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770693+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:59:55.770693+0000","last_peered":"2024-09-21T02:59:55.770693+0000","last_clean":"2024-09-21T02:59:55.770693+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:59:55.770693+0000","last_undegraded":"2024-09-21T02:59:55.770693+0000","last_fullsized":"2024-09-21T02:59:55.770693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247844+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:59:55.247844+0000","last_peered":"2024-09-21T02:59:55.247844+0000","last_clean":"2024-09-21T02:59:55.247844+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:59:55.247844+0000","last_undegraded":"2024-09-21T02:59:55.247844+0000","last_fullsized":"2024-09-21T02:59:55.247844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771806+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:59:55.771806+0000","last_peered":"2024-09-21T02:59:55.771806+0000","last_clean":"2024-09-21T02:59:55.771806+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:59:55.771806+0000","last_undegraded":"2024-09-21T02:59:55.771806+0000","last_fullsized":"2024-09-21T02:59:55.771806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769846+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:59:55.769846+0000","last_peered":"2024-09-21T02:59:55.769846+0000","last_clean":"2024-09-21T02:59:55.769846+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:59:55.769846+0000","last_undegraded":"2024-09-21T02:59:55.769846+0000","last_fullsized":"2024-09-21T02:59:55.769846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771385+0000","last_change":"2024-09-21T02:58:54.914527+0000","last_active":"2024-09-21T02:59:55.771385+0000","last_peered":"2024-09-21T02:59:55.771385+0000","last_clean":"2024-09-21T02:59:55.771385+0000","last_became_active":"2024-09-21T02:58:54.910405+0000","last_became_peered":"2024-09-21T02:58:54.910405+0000","last_unstale":"2024-09-21T02:59:55.771385+0000","last_undegraded":"2024-09-21T02:59:55.771385+0000","last_fullsized":"2024-09-21T02:59:55.771385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:50.419816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466388+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:59:55.466388+0000","last_peered":"2024-09-21T02:59:55.466388+0000","last_clean":"2024-09-21T02:59:55.466388+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:59:55.466388+0000","last_undegraded":"2024-09-21T02:59:55.466388+0000","last_fullsized":"2024-09-21T02:59:55.466388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245525+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:59:55.245525+0000","last_peered":"2024-09-21T02:59:55.245525+0000","last_clean":"2024-09-21T02:59:55.245525+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:59:55.245525+0000","last_undegraded":"2024-09-21T02:59:55.245525+0000","last_fullsized":"2024-09-21T02:59:55.245525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771799+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:59:55.771799+0000","last_peered":"2024-09-21T02:59:55.771799+0000","last_clean":"2024-09-21T02:59:55.771799+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:59:55.771799+0000","last_undegraded":"2024-09-21T02:59:55.771799+0000","last_fullsized":"2024-09-21T02:59:55.771799+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770339+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:59:55.770339+0000","last_peered":"2024-09-21T02:59:55.770339+0000","last_clean":"2024-09-21T02:59:55.770339+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:59:55.770339+0000","last_undegraded":"2024-09-21T02:59:55.770339+0000","last_fullsized":"2024-09-21T02:59:55.770339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247332+0000","last_change":"2024-09-21T02:58:54.918104+0000","last_active":"2024-09-21T02:59:55.247332+0000","last_peered":"2024-09-21T02:59:55.247332+0000","last_clean":"2024-09-21T02:59:55.247332+0000","last_became_active":"2024-09-21T02:58:54.911376+0000","last_became_peered":"2024-09-21T02:58:54.911376+0000","last_unstale":"2024-09-21T02:59:55.247332+0000","last_undegraded":"2024-09-21T02:59:55.247332+0000","last_fullsized":"2024-09-21T02:59:55.247332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:42.992181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797210+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:59:55.797210+0000","last_peered":"2024-09-21T02:59:55.797210+0000","last_clean":"2024-09-21T02:59:55.797210+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:59:55.797210+0000","last_undegraded":"2024-09-21T02:59:55.797210+0000","last_fullsized":"2024-09-21T02:59:55.797210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770242+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:59:55.770242+0000","last_peered":"2024-09-21T02:59:55.770242+0000","last_clean":"2024-09-21T02:59:55.770242+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:59:55.770242+0000","last_undegraded":"2024-09-21T02:59:55.770242+0000","last_fullsized":"2024-09-21T02:59:55.770242+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770500+0000","last_change":"2024-09-21T02:58:54.743356+0000","last_active":"2024-09-21T02:59:55.770500+0000","last_peered":"2024-09-21T02:59:55.770500+0000","last_clean":"2024-09-21T02:59:55.770500+0000","last_became_active":"2024-09-21T02:58:54.742875+0000","last_became_peered":"2024-09-21T02:58:54.742875+0000","last_unstale":"2024-09-21T02:59:55.770500+0000","last_undegraded":"2024-09-21T02:59:55.770500+0000","last_fullsized":"2024-09-21T02:59:55.770500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:47.202904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770077+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:59:55.770077+0000","last_peered":"2024-09-21T02:59:55.770077+0000","last_clean":"2024-09-21T02:59:55.770077+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:59:55.770077+0000","last_undegraded":"2024-09-21T02:59:55.770077+0000","last_fullsized":"2024-09-21T02:59:55.770077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":195,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743457,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":430015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41028,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704124,"statfs":{"total":95995035648,"available":95953022976,"internally_reserved":0,"allocated":1556480,"data_stored":1087900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":430597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":430619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1470464,"data_stored":1021082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":872448,"data_stored":430132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1478656,"data_stored":1020682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":872448,"data_stored":430037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":430353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":497563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":497221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705752,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":430105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:00:07.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 -- 172.21.15.16:0/2681551388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f54102980 msgr2=0x7f0f54102d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2681551388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f54102980 0x7f0f54102d90 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f0f44009900 tx=0x7f0f4402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 -- 172.21.15.16:0/2681551388 shutdown_connections 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2681551388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 0x7f0f5410f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2681551388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f541082b0 0x7f0f541086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2681551388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f54102980 0x7f0f54102d90 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 -- 172.21.15.16:0/2681551388 >> 172.21.15.16:0/2681551388 conn(0x7f0f540fe380 msgr2=0x7f0f541007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 -- 172.21.15.16:0/2681551388 shutdown_connections 2024-09-21T03:00:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.401+0000 7f0f5a751640 1 -- 172.21.15.16:0/2681551388 wait complete. 2024-09-21T03:00:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 Processor -- start 2024-09-21T03:00:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 -- start start 2024-09-21T03:00:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f54102980 0x7f0f541993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 0x7f0f5419ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54112910 con 0x7f0f541082b0 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54112790 con 0x7f0f54108c00 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.402+0000 7f0f5a751640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0f54112a90 con 0x7f0f54102980 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49600/0 (socket says 172.21.15.16:49600) 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 -- 172.21.15.16:0/2101627627 learned_addr learned my addr 172.21.15.16:0/2101627627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f53fff640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f54102980 0x7f0f541993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f58cee640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 0x7f0f5419ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 -- 172.21.15.16:0/2101627627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f54102980 msgr2=0x7f0f541993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f54102980 0x7f0f541993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 -- 172.21.15.16:0/2101627627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 msgr2=0x7f0f5419ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 0x7f0f5419ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 -- 172.21.15.16:0/2101627627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0f5419f250 con 0x7f0f541082b0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f537fe640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f4800ccc0 con 0x7f0f541082b0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0f480054d0 con 0x7f0f541082b0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0f48010430 con 0x7f0f541082b0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.403+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0f5419f540 con 0x7f0f541082b0 2024-09-21T03:00:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.404+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0f541a5d60 con 0x7f0f541082b0 2024-09-21T03:00:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.405+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0f54103bf0 con 0x7f0f541082b0 2024-09-21T03:00:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.406+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f0f48005000 con 0x7f0f541082b0 2024-09-21T03:00:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.407+0000 7f0f517fa640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 0x7f0f3c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.408+0000 7f0f53fff640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 0x7f0f3c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.408+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f0f480e7920 con 0x7f0f541082b0 2024-09-21T03:00:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.408+0000 7f0f53fff640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 0x7f0f3c0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f44005ab0 tx=0x7f0f44005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.410+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0f48056710 con 0x7f0f541082b0 2024-09-21T03:00:07.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.596+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0f54063800 con 0x7f0f3c0b6560 2024-09-21T03:00:07.601 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.600+0000 7f0f517fa640 1 -- 172.21.15.16:0/2101627627 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258731 (secure 0 0 0) 0x7f0f54063800 con 0x7f0f3c0b6560 2024-09-21T03:00:07.604 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:07.605 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 msgr2=0x7f0f3c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 0x7f0f3c0b8a70 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f0f44005ab0 tx=0x7f0f44005a20 comp rx=0 tx=0).stop 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 msgr2=0x7f0f54199950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800e9c0 tx=0x7f0f4800ee80 comp rx=0 tx=0).stop 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 shutdown_connections 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0f3c0b6560 0x7f0f3c0b8a70 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0f541082b0 0x7f0f54199950 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0f54108c00 0x7f0f5419ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 --2- 172.21.15.16:0/2101627627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0f54102980 0x7f0f541993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 >> 172.21.15.16:0/2101627627 conn(0x7f0f540fe380 msgr2=0x7f0f541078e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 shutdown_connections 2024-09-21T03:00:07.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.604+0000 7f0f5a751640 1 -- 172.21.15.16:0/2101627627 wait complete. 2024-09-21T03:00:07.617 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T03:00:07.173170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390668,"kb_used_data":12228,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551156,"statfs":{"total":1151940427776,"available":1151540383744,"internally_reserved":0,"allocated":12521472,"data_stored":7136306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.933954"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247799+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:59:55.247799+0000","last_peered":"2024-09-21T02:59:55.247799+0000","last_clean":"2024-09-21T02:59:55.247799+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:59:55.247799+0000","last_undegraded":"2024-09-21T02:59:55.247799+0000","last_fullsized":"2024-09-21T02:59:55.247799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796617+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:59:55.796617+0000","last_peered":"2024-09-21T02:59:55.796617+0000","last_clean":"2024-09-21T02:59:55.796617+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:59:55.796617+0000","last_undegraded":"2024-09-21T02:59:55.796617+0000","last_fullsized":"2024-09-21T02:59:55.796617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796611+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:59:55.796611+0000","last_peered":"2024-09-21T02:59:55.796611+0000","last_clean":"2024-09-21T02:59:55.796611+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:59:55.796611+0000","last_undegraded":"2024-09-21T02:59:55.796611+0000","last_fullsized":"2024-09-21T02:59:55.796611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242650+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:59:55.242650+0000","last_peered":"2024-09-21T02:59:55.242650+0000","last_clean":"2024-09-21T02:59:55.242650+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:59:55.242650+0000","last_undegraded":"2024-09-21T02:59:55.242650+0000","last_fullsized":"2024-09-21T02:59:55.242650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769885+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:59:55.769885+0000","last_peered":"2024-09-21T02:59:55.769885+0000","last_clean":"2024-09-21T02:59:55.769885+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:59:55.769885+0000","last_undegraded":"2024-09-21T02:59:55.769885+0000","last_fullsized":"2024-09-21T02:59:55.769885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769866+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:59:55.769866+0000","last_peered":"2024-09-21T02:59:55.769866+0000","last_clean":"2024-09-21T02:59:55.769866+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:59:55.769866+0000","last_undegraded":"2024-09-21T02:59:55.769866+0000","last_fullsized":"2024-09-21T02:59:55.769866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466197+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:59:55.466197+0000","last_peered":"2024-09-21T02:59:55.466197+0000","last_clean":"2024-09-21T02:59:55.466197+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:59:55.466197+0000","last_undegraded":"2024-09-21T02:59:55.466197+0000","last_fullsized":"2024-09-21T02:59:55.466197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769683+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:59:55.769683+0000","last_peered":"2024-09-21T02:59:55.769683+0000","last_clean":"2024-09-21T02:59:55.769683+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:59:55.769683+0000","last_undegraded":"2024-09-21T02:59:55.769683+0000","last_fullsized":"2024-09-21T02:59:55.769683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769663+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:59:55.769663+0000","last_peered":"2024-09-21T02:59:55.769663+0000","last_clean":"2024-09-21T02:59:55.769663+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:59:55.769663+0000","last_undegraded":"2024-09-21T02:59:55.769663+0000","last_fullsized":"2024-09-21T02:59:55.769663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769926+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:59:55.769926+0000","last_peered":"2024-09-21T02:59:55.769926+0000","last_clean":"2024-09-21T02:59:55.769926+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:59:55.769926+0000","last_undegraded":"2024-09-21T02:59:55.769926+0000","last_fullsized":"2024-09-21T02:59:55.769926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466653+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:59:55.466653+0000","last_peered":"2024-09-21T02:59:55.466653+0000","last_clean":"2024-09-21T02:59:55.466653+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:59:55.466653+0000","last_undegraded":"2024-09-21T02:59:55.466653+0000","last_fullsized":"2024-09-21T02:59:55.466653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242827+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:59:55.242827+0000","last_peered":"2024-09-21T02:59:55.242827+0000","last_clean":"2024-09-21T02:59:55.242827+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:59:55.242827+0000","last_undegraded":"2024-09-21T02:59:55.242827+0000","last_fullsized":"2024-09-21T02:59:55.242827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241546+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:59:55.241546+0000","last_peered":"2024-09-21T02:59:55.241546+0000","last_clean":"2024-09-21T02:59:55.241546+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:59:55.241546+0000","last_undegraded":"2024-09-21T02:59:55.241546+0000","last_fullsized":"2024-09-21T02:59:55.241546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796764+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:59:55.796764+0000","last_peered":"2024-09-21T02:59:55.796764+0000","last_clean":"2024-09-21T02:59:55.796764+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:59:55.796764+0000","last_undegraded":"2024-09-21T02:59:55.796764+0000","last_fullsized":"2024-09-21T02:59:55.796764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796581+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:59:55.796581+0000","last_peered":"2024-09-21T02:59:55.796581+0000","last_clean":"2024-09-21T02:59:55.796581+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:59:55.796581+0000","last_undegraded":"2024-09-21T02:59:55.796581+0000","last_fullsized":"2024-09-21T02:59:55.796581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770150+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:59:55.770150+0000","last_peered":"2024-09-21T02:59:55.770150+0000","last_clean":"2024-09-21T02:59:55.770150+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:59:55.770150+0000","last_undegraded":"2024-09-21T02:59:55.770150+0000","last_fullsized":"2024-09-21T02:59:55.770150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247833+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:59:55.247833+0000","last_peered":"2024-09-21T02:59:55.247833+0000","last_clean":"2024-09-21T02:59:55.247833+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:59:55.247833+0000","last_undegraded":"2024-09-21T02:59:55.247833+0000","last_fullsized":"2024-09-21T02:59:55.247833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770586+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:59:55.770586+0000","last_peered":"2024-09-21T02:59:55.770586+0000","last_clean":"2024-09-21T02:59:55.770586+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:59:55.770586+0000","last_undegraded":"2024-09-21T02:59:55.770586+0000","last_fullsized":"2024-09-21T02:59:55.770586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770832+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:59:55.770832+0000","last_peered":"2024-09-21T02:59:55.770832+0000","last_clean":"2024-09-21T02:59:55.770832+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:59:55.770832+0000","last_undegraded":"2024-09-21T02:59:55.770832+0000","last_fullsized":"2024-09-21T02:59:55.770832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247315+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:59:55.247315+0000","last_peered":"2024-09-21T02:59:55.247315+0000","last_clean":"2024-09-21T02:59:55.247315+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:59:55.247315+0000","last_undegraded":"2024-09-21T02:59:55.247315+0000","last_fullsized":"2024-09-21T02:59:55.247315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241585+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:59:55.241585+0000","last_peered":"2024-09-21T02:59:55.241585+0000","last_clean":"2024-09-21T02:59:55.241585+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:59:55.241585+0000","last_undegraded":"2024-09-21T02:59:55.241585+0000","last_fullsized":"2024-09-21T02:59:55.241585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770370+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:59:55.770370+0000","last_peered":"2024-09-21T02:59:55.770370+0000","last_clean":"2024-09-21T02:59:55.770370+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:59:55.770370+0000","last_undegraded":"2024-09-21T02:59:55.770370+0000","last_fullsized":"2024-09-21T02:59:55.770370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770410+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:59:55.770410+0000","last_peered":"2024-09-21T02:59:55.770410+0000","last_clean":"2024-09-21T02:59:55.770410+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:59:55.770410+0000","last_undegraded":"2024-09-21T02:59:55.770410+0000","last_fullsized":"2024-09-21T02:59:55.770410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245184+0000","last_change":"2024-09-21T02:58:54.745262+0000","last_active":"2024-09-21T02:59:55.245184+0000","last_peered":"2024-09-21T02:59:55.245184+0000","last_clean":"2024-09-21T02:59:55.245184+0000","last_became_active":"2024-09-21T02:58:54.744809+0000","last_became_peered":"2024-09-21T02:58:54.744809+0000","last_unstale":"2024-09-21T02:59:55.245184+0000","last_undegraded":"2024-09-21T02:59:55.245184+0000","last_fullsized":"2024-09-21T02:59:55.245184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:32.627252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770274+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:59:55.770274+0000","last_peered":"2024-09-21T02:59:55.770274+0000","last_clean":"2024-09-21T02:59:55.770274+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:59:55.770274+0000","last_undegraded":"2024-09-21T02:59:55.770274+0000","last_fullsized":"2024-09-21T02:59:55.770274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770897+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:59:55.770897+0000","last_peered":"2024-09-21T02:59:55.770897+0000","last_clean":"2024-09-21T02:59:55.770897+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:59:55.770897+0000","last_undegraded":"2024-09-21T02:59:55.770897+0000","last_fullsized":"2024-09-21T02:59:55.770897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247598+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:59:55.247598+0000","last_peered":"2024-09-21T02:59:55.247598+0000","last_clean":"2024-09-21T02:59:55.247598+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:59:55.247598+0000","last_undegraded":"2024-09-21T02:59:55.247598+0000","last_fullsized":"2024-09-21T02:59:55.247598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241323+0000","last_change":"2024-09-21T02:58:54.744120+0000","last_active":"2024-09-21T02:59:55.241323+0000","last_peered":"2024-09-21T02:59:55.241323+0000","last_clean":"2024-09-21T02:59:55.241323+0000","last_became_active":"2024-09-21T02:58:54.743669+0000","last_became_peered":"2024-09-21T02:58:54.743669+0000","last_unstale":"2024-09-21T02:59:55.241323+0000","last_undegraded":"2024-09-21T02:59:55.241323+0000","last_fullsized":"2024-09-21T02:59:55.241323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:16.956302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796609+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:59:55.796609+0000","last_peered":"2024-09-21T02:59:55.796609+0000","last_clean":"2024-09-21T02:59:55.796609+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:59:55.796609+0000","last_undegraded":"2024-09-21T02:59:55.796609+0000","last_fullsized":"2024-09-21T02:59:55.796609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770512+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:59:55.770512+0000","last_peered":"2024-09-21T02:59:55.770512+0000","last_clean":"2024-09-21T02:59:55.770512+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:59:55.770512+0000","last_undegraded":"2024-09-21T02:59:55.770512+0000","last_fullsized":"2024-09-21T02:59:55.770512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770484+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:59:55.770484+0000","last_peered":"2024-09-21T02:59:55.770484+0000","last_clean":"2024-09-21T02:59:55.770484+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:59:55.770484+0000","last_undegraded":"2024-09-21T02:59:55.770484+0000","last_fullsized":"2024-09-21T02:59:55.770484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247962+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:59:55.247962+0000","last_peered":"2024-09-21T02:59:55.247962+0000","last_clean":"2024-09-21T02:59:55.247962+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:59:55.247962+0000","last_undegraded":"2024-09-21T02:59:55.247962+0000","last_fullsized":"2024-09-21T02:59:55.247962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769880+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:59:55.769880+0000","last_peered":"2024-09-21T02:59:55.769880+0000","last_clean":"2024-09-21T02:59:55.769880+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:59:55.769880+0000","last_undegraded":"2024-09-21T02:59:55.769880+0000","last_fullsized":"2024-09-21T02:59:55.769880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245399+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:59:55.245399+0000","last_peered":"2024-09-21T02:59:55.245399+0000","last_clean":"2024-09-21T02:59:55.245399+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:59:55.245399+0000","last_undegraded":"2024-09-21T02:59:55.245399+0000","last_fullsized":"2024-09-21T02:59:55.245399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245319+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:59:55.245319+0000","last_peered":"2024-09-21T02:59:55.245319+0000","last_clean":"2024-09-21T02:59:55.245319+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:59:55.245319+0000","last_undegraded":"2024-09-21T02:59:55.245319+0000","last_fullsized":"2024-09-21T02:59:55.245319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770839+0000","last_change":"2024-09-21T02:58:54.912079+0000","last_active":"2024-09-21T02:59:55.770839+0000","last_peered":"2024-09-21T02:59:55.770839+0000","last_clean":"2024-09-21T02:59:55.770839+0000","last_became_active":"2024-09-21T02:58:54.911451+0000","last_became_peered":"2024-09-21T02:58:54.911451+0000","last_unstale":"2024-09-21T02:59:55.770839+0000","last_undegraded":"2024-09-21T02:59:55.770839+0000","last_fullsized":"2024-09-21T02:59:55.770839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:02.692841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240875+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:59:55.240875+0000","last_peered":"2024-09-21T02:59:55.240875+0000","last_clean":"2024-09-21T02:59:55.240875+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:59:55.240875+0000","last_undegraded":"2024-09-21T02:59:55.240875+0000","last_fullsized":"2024-09-21T02:59:55.240875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769478+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:59:55.769478+0000","last_peered":"2024-09-21T02:59:55.769478+0000","last_clean":"2024-09-21T02:59:55.769478+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:59:55.769478+0000","last_undegraded":"2024-09-21T02:59:55.769478+0000","last_fullsized":"2024-09-21T02:59:55.769478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.773488+0000","last_change":"2024-09-21T02:59:55.772969+0000","last_active":"2024-09-21T02:59:55.773488+0000","last_peered":"2024-09-21T02:59:55.773488+0000","last_clean":"2024-09-21T02:59:55.773488+0000","last_became_active":"2024-09-21T02:59:55.772407+0000","last_became_peered":"2024-09-21T02:59:55.772407+0000","last_unstale":"2024-09-21T02:59:55.773488+0000","last_undegraded":"2024-09-21T02:59:55.773488+0000","last_fullsized":"2024-09-21T02:59:55.773488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:18.002558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769606+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:59:55.769606+0000","last_peered":"2024-09-21T02:59:55.769606+0000","last_clean":"2024-09-21T02:59:55.769606+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:59:55.769606+0000","last_undegraded":"2024-09-21T02:59:55.769606+0000","last_fullsized":"2024-09-21T02:59:55.769606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771476+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:59:55.771476+0000","last_peered":"2024-09-21T02:59:55.771476+0000","last_clean":"2024-09-21T02:59:55.771476+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:59:55.771476+0000","last_undegraded":"2024-09-21T02:59:55.771476+0000","last_fullsized":"2024-09-21T02:59:55.771476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771071+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:59:55.771071+0000","last_peered":"2024-09-21T02:59:55.771071+0000","last_clean":"2024-09-21T02:59:55.771071+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:59:55.771071+0000","last_undegraded":"2024-09-21T02:59:55.771071+0000","last_fullsized":"2024-09-21T02:59:55.771071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466729+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:59:55.466729+0000","last_peered":"2024-09-21T02:59:55.466729+0000","last_clean":"2024-09-21T02:59:55.466729+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:59:55.466729+0000","last_undegraded":"2024-09-21T02:59:55.466729+0000","last_fullsized":"2024-09-21T02:59:55.466729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797346+0000","last_change":"2024-09-21T02:58:54.745298+0000","last_active":"2024-09-21T02:59:55.797346+0000","last_peered":"2024-09-21T02:59:55.797346+0000","last_clean":"2024-09-21T02:59:55.797346+0000","last_became_active":"2024-09-21T02:58:54.744775+0000","last_became_peered":"2024-09-21T02:58:54.744775+0000","last_unstale":"2024-09-21T02:59:55.797346+0000","last_undegraded":"2024-09-21T02:59:55.797346+0000","last_fullsized":"2024-09-21T02:59:55.797346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:14.405920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245494+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:59:55.245494+0000","last_peered":"2024-09-21T02:59:55.245494+0000","last_clean":"2024-09-21T02:59:55.245494+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:59:55.245494+0000","last_undegraded":"2024-09-21T02:59:55.245494+0000","last_fullsized":"2024-09-21T02:59:55.245494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769650+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:59:55.769650+0000","last_peered":"2024-09-21T02:59:55.769650+0000","last_clean":"2024-09-21T02:59:55.769650+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:59:55.769650+0000","last_undegraded":"2024-09-21T02:59:55.769650+0000","last_fullsized":"2024-09-21T02:59:55.769650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":247,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:00:03.830470+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T03:00:03.830470+0000","last_peered":"2024-09-21T03:00:03.830470+0000","last_clean":"2024-09-21T03:00:03.830470+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T03:00:03.830470+0000","last_undegraded":"2024-09-21T03:00:03.830470+0000","last_fullsized":"2024-09-21T03:00:03.830470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":160,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769962+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:59:55.769962+0000","last_peered":"2024-09-21T02:59:55.769962+0000","last_clean":"2024-09-21T02:59:55.769962+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:59:55.769962+0000","last_undegraded":"2024-09-21T02:59:55.769962+0000","last_fullsized":"2024-09-21T02:59:55.769962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771792+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:59:55.771792+0000","last_peered":"2024-09-21T02:59:55.771792+0000","last_clean":"2024-09-21T02:59:55.771792+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:59:55.771792+0000","last_undegraded":"2024-09-21T02:59:55.771792+0000","last_fullsized":"2024-09-21T02:59:55.771792+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240960+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:59:55.240960+0000","last_peered":"2024-09-21T02:59:55.240960+0000","last_clean":"2024-09-21T02:59:55.240960+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:59:55.240960+0000","last_undegraded":"2024-09-21T02:59:55.240960+0000","last_fullsized":"2024-09-21T02:59:55.240960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245476+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:59:55.245476+0000","last_peered":"2024-09-21T02:59:55.245476+0000","last_clean":"2024-09-21T02:59:55.245476+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:59:55.245476+0000","last_undegraded":"2024-09-21T02:59:55.245476+0000","last_fullsized":"2024-09-21T02:59:55.245476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466508+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:59:55.466508+0000","last_peered":"2024-09-21T02:59:55.466508+0000","last_clean":"2024-09-21T02:59:55.466508+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:59:55.466508+0000","last_undegraded":"2024-09-21T02:59:55.466508+0000","last_fullsized":"2024-09-21T02:59:55.466508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242473+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:59:55.242473+0000","last_peered":"2024-09-21T02:59:55.242473+0000","last_clean":"2024-09-21T02:59:55.242473+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:59:55.242473+0000","last_undegraded":"2024-09-21T02:59:55.242473+0000","last_fullsized":"2024-09-21T02:59:55.242473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466307+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:59:55.466307+0000","last_peered":"2024-09-21T02:59:55.466307+0000","last_clean":"2024-09-21T02:59:55.466307+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:59:55.466307+0000","last_undegraded":"2024-09-21T02:59:55.466307+0000","last_fullsized":"2024-09-21T02:59:55.466307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770297+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:59:55.770297+0000","last_peered":"2024-09-21T02:59:55.770297+0000","last_clean":"2024-09-21T02:59:55.770297+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:59:55.770297+0000","last_undegraded":"2024-09-21T02:59:55.770297+0000","last_fullsized":"2024-09-21T02:59:55.770297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245247+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:59:55.245247+0000","last_peered":"2024-09-21T02:59:55.245247+0000","last_clean":"2024-09-21T02:59:55.245247+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:59:55.245247+0000","last_undegraded":"2024-09-21T02:59:55.245247+0000","last_fullsized":"2024-09-21T02:59:55.245247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771747+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:59:55.771747+0000","last_peered":"2024-09-21T02:59:55.771747+0000","last_clean":"2024-09-21T02:59:55.771747+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:59:55.771747+0000","last_undegraded":"2024-09-21T02:59:55.771747+0000","last_fullsized":"2024-09-21T02:59:55.771747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247441+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:59:55.247441+0000","last_peered":"2024-09-21T02:59:55.247441+0000","last_clean":"2024-09-21T02:59:55.247441+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:59:55.247441+0000","last_undegraded":"2024-09-21T02:59:55.247441+0000","last_fullsized":"2024-09-21T02:59:55.247441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241296+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:59:55.241296+0000","last_peered":"2024-09-21T02:59:55.241296+0000","last_clean":"2024-09-21T02:59:55.241296+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:59:55.241296+0000","last_undegraded":"2024-09-21T02:59:55.241296+0000","last_fullsized":"2024-09-21T02:59:55.241296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466773+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:59:55.466773+0000","last_peered":"2024-09-21T02:59:55.466773+0000","last_clean":"2024-09-21T02:59:55.466773+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:59:55.466773+0000","last_undegraded":"2024-09-21T02:59:55.466773+0000","last_fullsized":"2024-09-21T02:59:55.466773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245695+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:59:55.245695+0000","last_peered":"2024-09-21T02:59:55.245695+0000","last_clean":"2024-09-21T02:59:55.245695+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:59:55.245695+0000","last_undegraded":"2024-09-21T02:59:55.245695+0000","last_fullsized":"2024-09-21T02:59:55.245695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466573+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:59:55.466573+0000","last_peered":"2024-09-21T02:59:55.466573+0000","last_clean":"2024-09-21T02:59:55.466573+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:59:55.466573+0000","last_undegraded":"2024-09-21T02:59:55.466573+0000","last_fullsized":"2024-09-21T02:59:55.466573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.772036+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:59:55.772036+0000","last_peered":"2024-09-21T02:59:55.772036+0000","last_clean":"2024-09-21T02:59:55.772036+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:59:55.772036+0000","last_undegraded":"2024-09-21T02:59:55.772036+0000","last_fullsized":"2024-09-21T02:59:55.772036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466818+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:59:55.466818+0000","last_peered":"2024-09-21T02:59:55.466818+0000","last_clean":"2024-09-21T02:59:55.466818+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:59:55.466818+0000","last_undegraded":"2024-09-21T02:59:55.466818+0000","last_fullsized":"2024-09-21T02:59:55.466818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466691+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:59:55.466691+0000","last_peered":"2024-09-21T02:59:55.466691+0000","last_clean":"2024-09-21T02:59:55.466691+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:59:55.466691+0000","last_undegraded":"2024-09-21T02:59:55.466691+0000","last_fullsized":"2024-09-21T02:59:55.466691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242571+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:59:55.242571+0000","last_peered":"2024-09-21T02:59:55.242571+0000","last_clean":"2024-09-21T02:59:55.242571+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:59:55.242571+0000","last_undegraded":"2024-09-21T02:59:55.242571+0000","last_fullsized":"2024-09-21T02:59:55.242571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769526+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:59:55.769526+0000","last_peered":"2024-09-21T02:59:55.769526+0000","last_clean":"2024-09-21T02:59:55.769526+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:59:55.769526+0000","last_undegraded":"2024-09-21T02:59:55.769526+0000","last_fullsized":"2024-09-21T02:59:55.769526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466591+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:59:55.466591+0000","last_peered":"2024-09-21T02:59:55.466591+0000","last_clean":"2024-09-21T02:59:55.466591+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:59:55.466591+0000","last_undegraded":"2024-09-21T02:59:55.466591+0000","last_fullsized":"2024-09-21T02:59:55.466591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240903+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:59:55.240903+0000","last_peered":"2024-09-21T02:59:55.240903+0000","last_clean":"2024-09-21T02:59:55.240903+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:59:55.240903+0000","last_undegraded":"2024-09-21T02:59:55.240903+0000","last_fullsized":"2024-09-21T02:59:55.240903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241097+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:59:55.241097+0000","last_peered":"2024-09-21T02:59:55.241097+0000","last_clean":"2024-09-21T02:59:55.241097+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:59:55.241097+0000","last_undegraded":"2024-09-21T02:59:55.241097+0000","last_fullsized":"2024-09-21T02:59:55.241097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770245+0000","last_change":"2024-09-21T02:58:54.909889+0000","last_active":"2024-09-21T02:59:55.770245+0000","last_peered":"2024-09-21T02:59:55.770245+0000","last_clean":"2024-09-21T02:59:55.770245+0000","last_became_active":"2024-09-21T02:58:54.909607+0000","last_became_peered":"2024-09-21T02:58:54.909607+0000","last_unstale":"2024-09-21T02:59:55.770245+0000","last_undegraded":"2024-09-21T02:59:55.770245+0000","last_fullsized":"2024-09-21T02:59:55.770245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466532+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:59:55.466532+0000","last_peered":"2024-09-21T02:59:55.466532+0000","last_clean":"2024-09-21T02:59:55.466532+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:59:55.466532+0000","last_undegraded":"2024-09-21T02:59:55.466532+0000","last_fullsized":"2024-09-21T02:59:55.466532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769978+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:59:55.769978+0000","last_peered":"2024-09-21T02:59:55.769978+0000","last_clean":"2024-09-21T02:59:55.769978+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:59:55.769978+0000","last_undegraded":"2024-09-21T02:59:55.769978+0000","last_fullsized":"2024-09-21T02:59:55.769978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770430+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:59:55.770430+0000","last_peered":"2024-09-21T02:59:55.770430+0000","last_clean":"2024-09-21T02:59:55.770430+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:59:55.770430+0000","last_undegraded":"2024-09-21T02:59:55.770430+0000","last_fullsized":"2024-09-21T02:59:55.770430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769861+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:59:55.769861+0000","last_peered":"2024-09-21T02:59:55.769861+0000","last_clean":"2024-09-21T02:59:55.769861+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:59:55.769861+0000","last_undegraded":"2024-09-21T02:59:55.769861+0000","last_fullsized":"2024-09-21T02:59:55.769861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771522+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:59:55.771522+0000","last_peered":"2024-09-21T02:59:55.771522+0000","last_clean":"2024-09-21T02:59:55.771522+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:59:55.771522+0000","last_undegraded":"2024-09-21T02:59:55.771522+0000","last_fullsized":"2024-09-21T02:59:55.771522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241371+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:59:55.241371+0000","last_peered":"2024-09-21T02:59:55.241371+0000","last_clean":"2024-09-21T02:59:55.241371+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:59:55.241371+0000","last_undegraded":"2024-09-21T02:59:55.241371+0000","last_fullsized":"2024-09-21T02:59:55.241371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770670+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:59:55.770670+0000","last_peered":"2024-09-21T02:59:55.770670+0000","last_clean":"2024-09-21T02:59:55.770670+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:59:55.770670+0000","last_undegraded":"2024-09-21T02:59:55.770670+0000","last_fullsized":"2024-09-21T02:59:55.770670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770035+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:59:55.770035+0000","last_peered":"2024-09-21T02:59:55.770035+0000","last_clean":"2024-09-21T02:59:55.770035+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:59:55.770035+0000","last_undegraded":"2024-09-21T02:59:55.770035+0000","last_fullsized":"2024-09-21T02:59:55.770035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242807+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:59:55.242807+0000","last_peered":"2024-09-21T02:59:55.242807+0000","last_clean":"2024-09-21T02:59:55.242807+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:59:55.242807+0000","last_undegraded":"2024-09-21T02:59:55.242807+0000","last_fullsized":"2024-09-21T02:59:55.242807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797033+0000","last_change":"2024-09-21T02:58:54.743546+0000","last_active":"2024-09-21T02:59:55.797033+0000","last_peered":"2024-09-21T02:59:55.797033+0000","last_clean":"2024-09-21T02:59:55.797033+0000","last_became_active":"2024-09-21T02:58:54.743107+0000","last_became_peered":"2024-09-21T02:58:54.743107+0000","last_unstale":"2024-09-21T02:59:55.797033+0000","last_undegraded":"2024-09-21T02:59:55.797033+0000","last_fullsized":"2024-09-21T02:59:55.797033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.458721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770566+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:59:55.770566+0000","last_peered":"2024-09-21T02:59:55.770566+0000","last_clean":"2024-09-21T02:59:55.770566+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:59:55.770566+0000","last_undegraded":"2024-09-21T02:59:55.770566+0000","last_fullsized":"2024-09-21T02:59:55.770566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769867+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:59:55.769867+0000","last_peered":"2024-09-21T02:59:55.769867+0000","last_clean":"2024-09-21T02:59:55.769867+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:59:55.769867+0000","last_undegraded":"2024-09-21T02:59:55.769867+0000","last_fullsized":"2024-09-21T02:59:55.769867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770693+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:59:55.770693+0000","last_peered":"2024-09-21T02:59:55.770693+0000","last_clean":"2024-09-21T02:59:55.770693+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:59:55.770693+0000","last_undegraded":"2024-09-21T02:59:55.770693+0000","last_fullsized":"2024-09-21T02:59:55.770693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247844+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:59:55.247844+0000","last_peered":"2024-09-21T02:59:55.247844+0000","last_clean":"2024-09-21T02:59:55.247844+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:59:55.247844+0000","last_undegraded":"2024-09-21T02:59:55.247844+0000","last_fullsized":"2024-09-21T02:59:55.247844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771806+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:59:55.771806+0000","last_peered":"2024-09-21T02:59:55.771806+0000","last_clean":"2024-09-21T02:59:55.771806+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:59:55.771806+0000","last_undegraded":"2024-09-21T02:59:55.771806+0000","last_fullsized":"2024-09-21T02:59:55.771806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769846+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:59:55.769846+0000","last_peered":"2024-09-21T02:59:55.769846+0000","last_clean":"2024-09-21T02:59:55.769846+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:59:55.769846+0000","last_undegraded":"2024-09-21T02:59:55.769846+0000","last_fullsized":"2024-09-21T02:59:55.769846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771385+0000","last_change":"2024-09-21T02:58:54.914527+0000","last_active":"2024-09-21T02:59:55.771385+0000","last_peered":"2024-09-21T02:59:55.771385+0000","last_clean":"2024-09-21T02:59:55.771385+0000","last_became_active":"2024-09-21T02:58:54.910405+0000","last_became_peered":"2024-09-21T02:58:54.910405+0000","last_unstale":"2024-09-21T02:59:55.771385+0000","last_undegraded":"2024-09-21T02:59:55.771385+0000","last_fullsized":"2024-09-21T02:59:55.771385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:50.419816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466388+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:59:55.466388+0000","last_peered":"2024-09-21T02:59:55.466388+0000","last_clean":"2024-09-21T02:59:55.466388+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:59:55.466388+0000","last_undegraded":"2024-09-21T02:59:55.466388+0000","last_fullsized":"2024-09-21T02:59:55.466388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245525+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:59:55.245525+0000","last_peered":"2024-09-21T02:59:55.245525+0000","last_clean":"2024-09-21T02:59:55.245525+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:59:55.245525+0000","last_undegraded":"2024-09-21T02:59:55.245525+0000","last_fullsized":"2024-09-21T02:59:55.245525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771799+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:59:55.771799+0000","last_peered":"2024-09-21T02:59:55.771799+0000","last_clean":"2024-09-21T02:59:55.771799+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:59:55.771799+0000","last_undegraded":"2024-09-21T02:59:55.771799+0000","last_fullsized":"2024-09-21T02:59:55.771799+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770339+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:59:55.770339+0000","last_peered":"2024-09-21T02:59:55.770339+0000","last_clean":"2024-09-21T02:59:55.770339+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:59:55.770339+0000","last_undegraded":"2024-09-21T02:59:55.770339+0000","last_fullsized":"2024-09-21T02:59:55.770339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247332+0000","last_change":"2024-09-21T02:58:54.918104+0000","last_active":"2024-09-21T02:59:55.247332+0000","last_peered":"2024-09-21T02:59:55.247332+0000","last_clean":"2024-09-21T02:59:55.247332+0000","last_became_active":"2024-09-21T02:58:54.911376+0000","last_became_peered":"2024-09-21T02:58:54.911376+0000","last_unstale":"2024-09-21T02:59:55.247332+0000","last_undegraded":"2024-09-21T02:59:55.247332+0000","last_fullsized":"2024-09-21T02:59:55.247332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:42.992181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797210+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:59:55.797210+0000","last_peered":"2024-09-21T02:59:55.797210+0000","last_clean":"2024-09-21T02:59:55.797210+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:59:55.797210+0000","last_undegraded":"2024-09-21T02:59:55.797210+0000","last_fullsized":"2024-09-21T02:59:55.797210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770242+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:59:55.770242+0000","last_peered":"2024-09-21T02:59:55.770242+0000","last_clean":"2024-09-21T02:59:55.770242+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:59:55.770242+0000","last_undegraded":"2024-09-21T02:59:55.770242+0000","last_fullsized":"2024-09-21T02:59:55.770242+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770500+0000","last_change":"2024-09-21T02:58:54.743356+0000","last_active":"2024-09-21T02:59:55.770500+0000","last_peered":"2024-09-21T02:59:55.770500+0000","last_clean":"2024-09-21T02:59:55.770500+0000","last_became_active":"2024-09-21T02:58:54.742875+0000","last_became_peered":"2024-09-21T02:58:54.742875+0000","last_unstale":"2024-09-21T02:59:55.770500+0000","last_undegraded":"2024-09-21T02:59:55.770500+0000","last_fullsized":"2024-09-21T02:59:55.770500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:47.202904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770077+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:59:55.770077+0000","last_peered":"2024-09-21T02:59:55.770077+0000","last_clean":"2024-09-21T02:59:55.770077+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:59:55.770077+0000","last_undegraded":"2024-09-21T02:59:55.770077+0000","last_fullsized":"2024-09-21T02:59:55.770077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":195,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743457,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":430015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41028,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704124,"statfs":{"total":95995035648,"available":95953022976,"internally_reserved":0,"allocated":1556480,"data_stored":1087900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":430597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":430619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1470464,"data_stored":1021082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":872448,"data_stored":430132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1478656,"data_stored":1020682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":872448,"data_stored":430037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":430353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":497563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":497221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705752,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":430105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:00:07.620 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-09-21T03:00:07.620 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 -- 172.21.15.16:0/1771822412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980109d30 msgr2=0x7f4980110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 --2- 172.21.15.16:0/1771822412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980109d30 0x7f4980110560 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f4978009900 tx=0x7f497802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 -- 172.21.15.16:0/1771822412 shutdown_connections 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 --2- 172.21.15.16:0/1771822412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980109d30 0x7f4980110560 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 --2- 172.21.15.16:0/1771822412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f49801093e0 0x7f49801097f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 --2- 172.21.15.16:0/1771822412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4980103ab0 0x7f4980103ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 -- 172.21.15.16:0/1771822412 >> 172.21.15.16:0/1771822412 conn(0x7f49800ff4d0 msgr2=0x7f4980101920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 -- 172.21.15.16:0/1771822412 shutdown_connections 2024-09-21T03:00:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.769+0000 7f49882a4640 1 -- 172.21.15.16:0/1771822412 wait complete. 2024-09-21T03:00:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 Processor -- start 2024-09-21T03:00:07.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 -- start start 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801093e0 0x7f498019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4980109d30 0x7f498019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f49801139f0 con 0x7f4980103ab0 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4980113870 con 0x7f4980109d30 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f49882a4640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4980113b70 con 0x7f49801093e0 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f4986040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f4986040640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49606/0 (socket says 172.21.15.16:49606) 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f4986040640 1 -- 172.21.15.16:0/2105457970 learned_addr learned my addr 172.21.15.16:0/2105457970 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f498583f640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801093e0 0x7f498019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.770+0000 7f4986841640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4980109d30 0x7f498019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 -- 172.21.15.16:0/2105457970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801093e0 msgr2=0x7f498019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801093e0 0x7f498019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 -- 172.21.15.16:0/2105457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4980109d30 msgr2=0x7f498019fc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4980109d30 0x7f498019fc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 -- 172.21.15.16:0/2105457970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49801a0340 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f4986040640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f497000ea80 tx=0x7f497000ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f497000ce40 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49801a0630 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4970005020 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f49700116d0 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.771+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49801a6dc0 con 0x7f4980103ab0 2024-09-21T03:00:07.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.772+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4980103ec0 con 0x7f4980103ab0 2024-09-21T03:00:07.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.775+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f49700054d0 con 0x7f4980103ab0 2024-09-21T03:00:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.775+0000 7f497f7fe640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 0x7f49680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.775+0000 7f498583f640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 0x7f49680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.775+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f49700e7b10 con 0x7f4980103ab0 2024-09-21T03:00:07.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.776+0000 7f498583f640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 0x7f49680b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4980108ed0 tx=0x7f4974005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:07.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.778+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4970016030 con 0x7f4980103ab0 2024-09-21T03:00:07.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.961+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f4980063800 con 0x7f49680b6560 2024-09-21T03:00:07.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.966+0000 7f497f7fe640 1 -- 172.21.15.16:0/2105457970 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+258731 (secure 0 0 0) 0x7f4980063800 con 0x7f49680b6560 2024-09-21T03:00:07.970 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:07.971 INFO:teuthology.orchestra.run.smithi016.stderr:dumped all 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 msgr2=0x7f49680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 0x7f49680b8a70 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4980108ed0 tx=0x7f4974005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 msgr2=0x7f498019a410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f497000ea80 tx=0x7f497000ef40 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 shutdown_connections 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f49680b6560 0x7f49680b8a70 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4980109d30 0x7f498019fc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f49801093e0 0x7f498019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 --2- 172.21.15.16:0/2105457970 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4980103ab0 0x7f498019a410 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 >> 172.21.15.16:0/2105457970 conn(0x7f49800ff4d0 msgr2=0x7f49801018f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:07.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 shutdown_connections 2024-09-21T03:00:07.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:07.969+0000 7f49882a4640 1 -- 172.21.15.16:0/2105457970 wait complete. 2024-09-21T03:00:07.982 INFO:teuthology.orchestra.run.smithi016.stdout:{"pg_ready":true,"pg_map":{"version":314,"stamp":"2024-09-21T03:00:07.173170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":658914,"num_objects":26,"num_object_clones":0,"num_object_copies":78,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":26,"num_whiteouts":0,"num_read":276,"num_read_kb":235,"num_write":216,"num_write_kb":2250,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":12,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":668,"ondisk_log_size":668,"up":291,"acting":291,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390668,"kb_used_data":12228,"kb_used_omap":18,"kb_used_meta":378157,"kb_avail":1124551156,"statfs":{"total":1151940427776,"available":1151540383744,"internally_reserved":0,"allocated":12521472,"data_stored":7136306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19143,"internal_metadata":387233081},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":308,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":21,"num_read_kb":11,"num_write":3,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"11.933954"},"pg_stats":[{"pgid":"3.3f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247799+0000","last_change":"2024-09-21T02:58:27.664991+0000","last_active":"2024-09-21T02:59:55.247799+0000","last_peered":"2024-09-21T02:59:55.247799+0000","last_clean":"2024-09-21T02:59:55.247799+0000","last_became_active":"2024-09-21T02:58:27.664612+0000","last_became_peered":"2024-09-21T02:58:27.664612+0000","last_unstale":"2024-09-21T02:59:55.247799+0000","last_undegraded":"2024-09-21T02:59:55.247799+0000","last_fullsized":"2024-09-21T02:59:55.247799+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:03:55.111413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796617+0000","last_change":"2024-09-21T02:58:27.655455+0000","last_active":"2024-09-21T02:59:55.796617+0000","last_peered":"2024-09-21T02:59:55.796617+0000","last_clean":"2024-09-21T02:59:55.796617+0000","last_became_active":"2024-09-21T02:58:27.655005+0000","last_became_peered":"2024-09-21T02:58:27.655005+0000","last_unstale":"2024-09-21T02:59:55.796617+0000","last_undegraded":"2024-09-21T02:59:55.796617+0000","last_fullsized":"2024-09-21T02:59:55.796617+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:42:46.385748+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796611+0000","last_change":"2024-09-21T02:58:27.663000+0000","last_active":"2024-09-21T02:59:55.796611+0000","last_peered":"2024-09-21T02:59:55.796611+0000","last_clean":"2024-09-21T02:59:55.796611+0000","last_became_active":"2024-09-21T02:58:27.662552+0000","last_became_peered":"2024-09-21T02:58:27.662552+0000","last_unstale":"2024-09-21T02:59:55.796611+0000","last_undegraded":"2024-09-21T02:59:55.796611+0000","last_fullsized":"2024-09-21T02:59:55.796611+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:43:22.777020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242650+0000","last_change":"2024-09-21T02:58:27.659643+0000","last_active":"2024-09-21T02:59:55.242650+0000","last_peered":"2024-09-21T02:59:55.242650+0000","last_clean":"2024-09-21T02:59:55.242650+0000","last_became_active":"2024-09-21T02:58:27.659257+0000","last_became_peered":"2024-09-21T02:58:27.659257+0000","last_unstale":"2024-09-21T02:59:55.242650+0000","last_undegraded":"2024-09-21T02:59:55.242650+0000","last_fullsized":"2024-09-21T02:59:55.242650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:36:09.096278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"77'4","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769885+0000","last_change":"2024-09-21T02:58:27.652199+0000","last_active":"2024-09-21T02:59:55.769885+0000","last_peered":"2024-09-21T02:59:55.769885+0000","last_clean":"2024-09-21T02:59:55.769885+0000","last_became_active":"2024-09-21T02:58:27.651738+0000","last_became_peered":"2024-09-21T02:58:27.651738+0000","last_unstale":"2024-09-21T02:59:55.769885+0000","last_undegraded":"2024-09-21T02:59:55.769885+0000","last_fullsized":"2024-09-21T02:59:55.769885+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:29:37.230726+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769866+0000","last_change":"2024-09-21T02:58:27.654577+0000","last_active":"2024-09-21T02:59:55.769866+0000","last_peered":"2024-09-21T02:59:55.769866+0000","last_clean":"2024-09-21T02:59:55.769866+0000","last_became_active":"2024-09-21T02:58:27.654159+0000","last_became_peered":"2024-09-21T02:58:27.654159+0000","last_unstale":"2024-09-21T02:59:55.769866+0000","last_undegraded":"2024-09-21T02:59:55.769866+0000","last_fullsized":"2024-09-21T02:59:55.769866+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:19:33.523749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466197+0000","last_change":"2024-09-21T02:58:27.661958+0000","last_active":"2024-09-21T02:59:55.466197+0000","last_peered":"2024-09-21T02:59:55.466197+0000","last_clean":"2024-09-21T02:59:55.466197+0000","last_became_active":"2024-09-21T02:58:27.661523+0000","last_became_peered":"2024-09-21T02:58:27.661523+0000","last_unstale":"2024-09-21T02:59:55.466197+0000","last_undegraded":"2024-09-21T02:59:55.466197+0000","last_fullsized":"2024-09-21T02:59:55.466197+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:30:31.909669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769683+0000","last_change":"2024-09-21T02:58:27.657530+0000","last_active":"2024-09-21T02:59:55.769683+0000","last_peered":"2024-09-21T02:59:55.769683+0000","last_clean":"2024-09-21T02:59:55.769683+0000","last_became_active":"2024-09-21T02:58:27.657201+0000","last_became_peered":"2024-09-21T02:58:27.657201+0000","last_unstale":"2024-09-21T02:59:55.769683+0000","last_undegraded":"2024-09-21T02:59:55.769683+0000","last_fullsized":"2024-09-21T02:59:55.769683+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:09:16.112724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769663+0000","last_change":"2024-09-21T02:58:27.664340+0000","last_active":"2024-09-21T02:59:55.769663+0000","last_peered":"2024-09-21T02:59:55.769663+0000","last_clean":"2024-09-21T02:59:55.769663+0000","last_became_active":"2024-09-21T02:58:27.663658+0000","last_became_peered":"2024-09-21T02:58:27.663658+0000","last_unstale":"2024-09-21T02:59:55.769663+0000","last_undegraded":"2024-09-21T02:59:55.769663+0000","last_fullsized":"2024-09-21T02:59:55.769663+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:26.528822+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"77'16","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769926+0000","last_change":"2024-09-21T02:58:27.660867+0000","last_active":"2024-09-21T02:59:55.769926+0000","last_peered":"2024-09-21T02:59:55.769926+0000","last_clean":"2024-09-21T02:59:55.769926+0000","last_became_active":"2024-09-21T02:58:27.660480+0000","last_became_peered":"2024-09-21T02:58:27.660480+0000","last_unstale":"2024-09-21T02:59:55.769926+0000","last_undegraded":"2024-09-21T02:59:55.769926+0000","last_fullsized":"2024-09-21T02:59:55.769926+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:32.086924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466653+0000","last_change":"2024-09-21T02:58:27.657622+0000","last_active":"2024-09-21T02:59:55.466653+0000","last_peered":"2024-09-21T02:59:55.466653+0000","last_clean":"2024-09-21T02:59:55.466653+0000","last_became_active":"2024-09-21T02:58:27.657247+0000","last_became_peered":"2024-09-21T02:58:27.657247+0000","last_unstale":"2024-09-21T02:59:55.466653+0000","last_undegraded":"2024-09-21T02:59:55.466653+0000","last_fullsized":"2024-09-21T02:59:55.466653+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:34:55.722803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242827+0000","last_change":"2024-09-21T02:58:27.659009+0000","last_active":"2024-09-21T02:59:55.242827+0000","last_peered":"2024-09-21T02:59:55.242827+0000","last_clean":"2024-09-21T02:59:55.242827+0000","last_became_active":"2024-09-21T02:58:27.658502+0000","last_became_peered":"2024-09-21T02:58:27.658502+0000","last_unstale":"2024-09-21T02:59:55.242827+0000","last_undegraded":"2024-09-21T02:59:55.242827+0000","last_fullsized":"2024-09-21T02:59:55.242827+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:05:36.747434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"77'10","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241546+0000","last_change":"2024-09-21T02:58:27.649023+0000","last_active":"2024-09-21T02:59:55.241546+0000","last_peered":"2024-09-21T02:59:55.241546+0000","last_clean":"2024-09-21T02:59:55.241546+0000","last_became_active":"2024-09-21T02:58:27.648173+0000","last_became_peered":"2024-09-21T02:58:27.648173+0000","last_unstale":"2024-09-21T02:59:55.241546+0000","last_undegraded":"2024-09-21T02:59:55.241546+0000","last_fullsized":"2024-09-21T02:59:55.241546+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:29:56.555796+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796764+0000","last_change":"2024-09-21T02:58:27.653359+0000","last_active":"2024-09-21T02:59:55.796764+0000","last_peered":"2024-09-21T02:59:55.796764+0000","last_clean":"2024-09-21T02:59:55.796764+0000","last_became_active":"2024-09-21T02:58:27.653020+0000","last_became_peered":"2024-09-21T02:58:27.653020+0000","last_unstale":"2024-09-21T02:59:55.796764+0000","last_undegraded":"2024-09-21T02:59:55.796764+0000","last_fullsized":"2024-09-21T02:59:55.796764+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:52:03.784197+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796581+0000","last_change":"2024-09-21T02:58:27.660066+0000","last_active":"2024-09-21T02:59:55.796581+0000","last_peered":"2024-09-21T02:59:55.796581+0000","last_clean":"2024-09-21T02:59:55.796581+0000","last_became_active":"2024-09-21T02:58:27.659687+0000","last_became_peered":"2024-09-21T02:58:27.659687+0000","last_unstale":"2024-09-21T02:59:55.796581+0000","last_undegraded":"2024-09-21T02:59:55.796581+0000","last_fullsized":"2024-09-21T02:59:55.796581+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:57:17.775321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770150+0000","last_change":"2024-09-21T02:58:27.654649+0000","last_active":"2024-09-21T02:59:55.770150+0000","last_peered":"2024-09-21T02:59:55.770150+0000","last_clean":"2024-09-21T02:59:55.770150+0000","last_became_active":"2024-09-21T02:58:27.654204+0000","last_became_peered":"2024-09-21T02:58:27.654204+0000","last_unstale":"2024-09-21T02:59:55.770150+0000","last_undegraded":"2024-09-21T02:59:55.770150+0000","last_fullsized":"2024-09-21T02:59:55.770150+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:54:43.842946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247833+0000","last_change":"2024-09-21T02:58:27.665007+0000","last_active":"2024-09-21T02:59:55.247833+0000","last_peered":"2024-09-21T02:59:55.247833+0000","last_clean":"2024-09-21T02:59:55.247833+0000","last_became_active":"2024-09-21T02:58:27.664645+0000","last_became_peered":"2024-09-21T02:58:27.664645+0000","last_unstale":"2024-09-21T02:59:55.247833+0000","last_undegraded":"2024-09-21T02:59:55.247833+0000","last_fullsized":"2024-09-21T02:59:55.247833+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:13:03.757828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770586+0000","last_change":"2024-09-21T02:58:27.652002+0000","last_active":"2024-09-21T02:59:55.770586+0000","last_peered":"2024-09-21T02:59:55.770586+0000","last_clean":"2024-09-21T02:59:55.770586+0000","last_became_active":"2024-09-21T02:58:27.651672+0000","last_became_peered":"2024-09-21T02:58:27.651672+0000","last_unstale":"2024-09-21T02:59:55.770586+0000","last_undegraded":"2024-09-21T02:59:55.770586+0000","last_fullsized":"2024-09-21T02:59:55.770586+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:48.403763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"77'7","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770832+0000","last_change":"2024-09-21T02:58:27.656964+0000","last_active":"2024-09-21T02:59:55.770832+0000","last_peered":"2024-09-21T02:59:55.770832+0000","last_clean":"2024-09-21T02:59:55.770832+0000","last_became_active":"2024-09-21T02:58:27.656618+0000","last_became_peered":"2024-09-21T02:58:27.656618+0000","last_unstale":"2024-09-21T02:59:55.770832+0000","last_undegraded":"2024-09-21T02:59:55.770832+0000","last_fullsized":"2024-09-21T02:59:55.770832+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:21:16.313853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2c","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247315+0000","last_change":"2024-09-21T02:58:27.662888+0000","last_active":"2024-09-21T02:59:55.247315+0000","last_peered":"2024-09-21T02:59:55.247315+0000","last_clean":"2024-09-21T02:59:55.247315+0000","last_became_active":"2024-09-21T02:58:27.662499+0000","last_became_peered":"2024-09-21T02:58:27.662499+0000","last_unstale":"2024-09-21T02:59:55.247315+0000","last_undegraded":"2024-09-21T02:59:55.247315+0000","last_fullsized":"2024-09-21T02:59:55.247315+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:24:14.365277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2b","version":"77'11","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241585+0000","last_change":"2024-09-21T02:58:27.652324+0000","last_active":"2024-09-21T02:59:55.241585+0000","last_peered":"2024-09-21T02:59:55.241585+0000","last_clean":"2024-09-21T02:59:55.241585+0000","last_became_active":"2024-09-21T02:58:27.651930+0000","last_became_peered":"2024-09-21T02:58:27.651930+0000","last_unstale":"2024-09-21T02:59:55.241585+0000","last_undegraded":"2024-09-21T02:59:55.241585+0000","last_fullsized":"2024-09-21T02:59:55.241585+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:03:28.319259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2a","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770370+0000","last_change":"2024-09-21T02:58:27.654618+0000","last_active":"2024-09-21T02:59:55.770370+0000","last_peered":"2024-09-21T02:59:55.770370+0000","last_clean":"2024-09-21T02:59:55.770370+0000","last_became_active":"2024-09-21T02:58:27.654238+0000","last_became_peered":"2024-09-21T02:58:27.654238+0000","last_unstale":"2024-09-21T02:59:55.770370+0000","last_undegraded":"2024-09-21T02:59:55.770370+0000","last_fullsized":"2024-09-21T02:59:55.770370+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:06:10.499468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.29","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770410+0000","last_change":"2024-09-21T02:58:27.660325+0000","last_active":"2024-09-21T02:59:55.770410+0000","last_peered":"2024-09-21T02:59:55.770410+0000","last_clean":"2024-09-21T02:59:55.770410+0000","last_became_active":"2024-09-21T02:58:27.659928+0000","last_became_peered":"2024-09-21T02:58:27.659928+0000","last_unstale":"2024-09-21T02:59:55.770410+0000","last_undegraded":"2024-09-21T02:59:55.770410+0000","last_fullsized":"2024-09-21T02:59:55.770410+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:43:31.421025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"77'6","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245184+0000","last_change":"2024-09-21T02:58:54.745262+0000","last_active":"2024-09-21T02:59:55.245184+0000","last_peered":"2024-09-21T02:59:55.245184+0000","last_clean":"2024-09-21T02:59:55.245184+0000","last_became_active":"2024-09-21T02:58:54.744809+0000","last_became_peered":"2024-09-21T02:58:54.744809+0000","last_unstale":"2024-09-21T02:59:55.245184+0000","last_undegraded":"2024-09-21T02:59:55.245184+0000","last_fullsized":"2024-09-21T02:59:55.245184+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:56:32.627252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,8],"acting":[5,9,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"77'2","reported_seq":23,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770274+0000","last_change":"2024-09-21T02:58:27.653095+0000","last_active":"2024-09-21T02:59:55.770274+0000","last_peered":"2024-09-21T02:59:55.770274+0000","last_clean":"2024-09-21T02:59:55.770274+0000","last_became_active":"2024-09-21T02:58:27.652732+0000","last_became_peered":"2024-09-21T02:58:27.652732+0000","last_unstale":"2024-09-21T02:59:55.770274+0000","last_undegraded":"2024-09-21T02:59:55.770274+0000","last_fullsized":"2024-09-21T02:59:55.770274+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:57:13.164124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"77'15","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770897+0000","last_change":"2024-09-21T02:58:27.662228+0000","last_active":"2024-09-21T02:59:55.770897+0000","last_peered":"2024-09-21T02:59:55.770897+0000","last_clean":"2024-09-21T02:59:55.770897+0000","last_became_active":"2024-09-21T02:58:27.661592+0000","last_became_peered":"2024-09-21T02:58:27.661592+0000","last_unstale":"2024-09-21T02:59:55.770897+0000","last_undegraded":"2024-09-21T02:59:55.770897+0000","last_fullsized":"2024-09-21T02:59:55.770897+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:22:35.964641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.25","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247598+0000","last_change":"2024-09-21T02:58:27.667189+0000","last_active":"2024-09-21T02:59:55.247598+0000","last_peered":"2024-09-21T02:59:55.247598+0000","last_clean":"2024-09-21T02:59:55.247598+0000","last_became_active":"2024-09-21T02:58:27.666805+0000","last_became_peered":"2024-09-21T02:58:27.666805+0000","last_unstale":"2024-09-21T02:59:55.247598+0000","last_undegraded":"2024-09-21T02:59:55.247598+0000","last_fullsized":"2024-09-21T02:59:55.247598+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:32:03.112135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.24","version":"77'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241323+0000","last_change":"2024-09-21T02:58:54.744120+0000","last_active":"2024-09-21T02:59:55.241323+0000","last_peered":"2024-09-21T02:59:55.241323+0000","last_clean":"2024-09-21T02:59:55.241323+0000","last_became_active":"2024-09-21T02:58:54.743669+0000","last_became_peered":"2024-09-21T02:58:54.743669+0000","last_unstale":"2024-09-21T02:59:55.241323+0000","last_undegraded":"2024-09-21T02:59:55.241323+0000","last_fullsized":"2024-09-21T02:59:55.241323+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:53:16.956302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,9,0],"acting":[6,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.23","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.796609+0000","last_change":"2024-09-21T02:58:27.664554+0000","last_active":"2024-09-21T02:59:55.796609+0000","last_peered":"2024-09-21T02:59:55.796609+0000","last_clean":"2024-09-21T02:59:55.796609+0000","last_became_active":"2024-09-21T02:58:27.664147+0000","last_became_peered":"2024-09-21T02:58:27.664147+0000","last_unstale":"2024-09-21T02:59:55.796609+0000","last_undegraded":"2024-09-21T02:59:55.796609+0000","last_fullsized":"2024-09-21T02:59:55.796609+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:24:17.305674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770512+0000","last_change":"2024-09-21T02:58:27.658121+0000","last_active":"2024-09-21T02:59:55.770512+0000","last_peered":"2024-09-21T02:59:55.770512+0000","last_clean":"2024-09-21T02:59:55.770512+0000","last_became_active":"2024-09-21T02:58:27.657754+0000","last_became_peered":"2024-09-21T02:58:27.657754+0000","last_unstale":"2024-09-21T02:59:55.770512+0000","last_undegraded":"2024-09-21T02:59:55.770512+0000","last_fullsized":"2024-09-21T02:59:55.770512+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:52:31.793594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.21","version":"77'12","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770484+0000","last_change":"2024-09-21T02:58:27.665265+0000","last_active":"2024-09-21T02:59:55.770484+0000","last_peered":"2024-09-21T02:59:55.770484+0000","last_clean":"2024-09-21T02:59:55.770484+0000","last_became_active":"2024-09-21T02:58:27.664898+0000","last_became_peered":"2024-09-21T02:58:27.664898+0000","last_unstale":"2024-09-21T02:59:55.770484+0000","last_undegraded":"2024-09-21T02:59:55.770484+0000","last_fullsized":"2024-09-21T02:59:55.770484+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:57:51.252899+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247962+0000","last_change":"2024-09-21T02:58:27.666674+0000","last_active":"2024-09-21T02:59:55.247962+0000","last_peered":"2024-09-21T02:59:55.247962+0000","last_clean":"2024-09-21T02:59:55.247962+0000","last_became_active":"2024-09-21T02:58:27.666061+0000","last_became_peered":"2024-09-21T02:58:27.666061+0000","last_unstale":"2024-09-21T02:59:55.247962+0000","last_undegraded":"2024-09-21T02:59:55.247962+0000","last_fullsized":"2024-09-21T02:59:55.247962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:23:02.460635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1a","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769880+0000","last_change":"2024-09-21T02:58:27.655957+0000","last_active":"2024-09-21T02:59:55.769880+0000","last_peered":"2024-09-21T02:59:55.769880+0000","last_clean":"2024-09-21T02:59:55.769880+0000","last_became_active":"2024-09-21T02:58:27.655583+0000","last_became_peered":"2024-09-21T02:58:27.655583+0000","last_unstale":"2024-09-21T02:59:55.769880+0000","last_undegraded":"2024-09-21T02:59:55.769880+0000","last_fullsized":"2024-09-21T02:59:55.769880+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:15:20.214485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.1b","version":"75'1","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245399+0000","last_change":"2024-09-21T02:58:25.638701+0000","last_active":"2024-09-21T02:59:55.245399+0000","last_peered":"2024-09-21T02:59:55.245399+0000","last_clean":"2024-09-21T02:59:55.245399+0000","last_became_active":"2024-09-21T02:58:25.638254+0000","last_became_peered":"2024-09-21T02:58:25.638254+0000","last_unstale":"2024-09-21T02:59:55.245399+0000","last_undegraded":"2024-09-21T02:59:55.245399+0000","last_fullsized":"2024-09-21T02:59:55.245399+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:06.793493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1b","version":"77'8","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245319+0000","last_change":"2024-09-21T02:58:27.645153+0000","last_active":"2024-09-21T02:59:55.245319+0000","last_peered":"2024-09-21T02:59:55.245319+0000","last_clean":"2024-09-21T02:59:55.245319+0000","last_became_active":"2024-09-21T02:58:27.644729+0000","last_became_peered":"2024-09-21T02:58:27.644729+0000","last_unstale":"2024-09-21T02:59:55.245319+0000","last_undegraded":"2024-09-21T02:59:55.245319+0000","last_fullsized":"2024-09-21T02:59:55.245319+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:16:59.994035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1a","version":"75'1","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770839+0000","last_change":"2024-09-21T02:58:54.912079+0000","last_active":"2024-09-21T02:59:55.770839+0000","last_peered":"2024-09-21T02:59:55.770839+0000","last_clean":"2024-09-21T02:59:55.770839+0000","last_became_active":"2024-09-21T02:58:54.911451+0000","last_became_peered":"2024-09-21T02:58:54.911451+0000","last_unstale":"2024-09-21T02:59:55.770839+0000","last_undegraded":"2024-09-21T02:59:55.770839+0000","last_fullsized":"2024-09-21T02:59:55.770839+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:06:02.692841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,6],"acting":[10,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240875+0000","last_change":"2024-09-21T02:58:25.632638+0000","last_active":"2024-09-21T02:59:55.240875+0000","last_peered":"2024-09-21T02:59:55.240875+0000","last_clean":"2024-09-21T02:59:55.240875+0000","last_became_active":"2024-09-21T02:58:25.632223+0000","last_became_peered":"2024-09-21T02:58:25.632223+0000","last_unstale":"2024-09-21T02:59:55.240875+0000","last_undegraded":"2024-09-21T02:59:55.240875+0000","last_fullsized":"2024-09-21T02:59:55.240875+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:57:21.051889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769478+0000","last_change":"2024-09-21T02:58:27.650466+0000","last_active":"2024-09-21T02:59:55.769478+0000","last_peered":"2024-09-21T02:59:55.769478+0000","last_clean":"2024-09-21T02:59:55.769478+0000","last_became_active":"2024-09-21T02:58:27.650096+0000","last_became_peered":"2024-09-21T02:58:27.650096+0000","last_unstale":"2024-09-21T02:59:55.769478+0000","last_undegraded":"2024-09-21T02:59:55.769478+0000","last_fullsized":"2024-09-21T02:59:55.769478+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:29.605682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.773488+0000","last_change":"2024-09-21T02:59:55.772969+0000","last_active":"2024-09-21T02:59:55.773488+0000","last_peered":"2024-09-21T02:59:55.773488+0000","last_clean":"2024-09-21T02:59:55.773488+0000","last_became_active":"2024-09-21T02:59:55.772407+0000","last_became_peered":"2024-09-21T02:59:55.772407+0000","last_unstale":"2024-09-21T02:59:55.773488+0000","last_undegraded":"2024-09-21T02:59:55.773488+0000","last_fullsized":"2024-09-21T02:59:55.773488+0000","mapping_epoch":78,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":79,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:02:18.002558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,7],"acting":[11,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.19","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769606+0000","last_change":"2024-09-21T02:58:27.649907+0000","last_active":"2024-09-21T02:59:55.769606+0000","last_peered":"2024-09-21T02:59:55.769606+0000","last_clean":"2024-09-21T02:59:55.769606+0000","last_became_active":"2024-09-21T02:58:27.649605+0000","last_became_peered":"2024-09-21T02:58:27.649605+0000","last_unstale":"2024-09-21T02:59:55.769606+0000","last_undegraded":"2024-09-21T02:59:55.769606+0000","last_fullsized":"2024-09-21T02:59:55.769606+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:54:13.797064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771476+0000","last_change":"2024-09-21T02:58:25.639266+0000","last_active":"2024-09-21T02:59:55.771476+0000","last_peered":"2024-09-21T02:59:55.771476+0000","last_clean":"2024-09-21T02:59:55.771476+0000","last_became_active":"2024-09-21T02:58:25.638859+0000","last_became_peered":"2024-09-21T02:58:25.638859+0000","last_unstale":"2024-09-21T02:59:55.771476+0000","last_undegraded":"2024-09-21T02:59:55.771476+0000","last_fullsized":"2024-09-21T02:59:55.771476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:42:18.559334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"77'10","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771071+0000","last_change":"2024-09-21T02:58:27.666540+0000","last_active":"2024-09-21T02:59:55.771071+0000","last_peered":"2024-09-21T02:59:55.771071+0000","last_clean":"2024-09-21T02:59:55.771071+0000","last_became_active":"2024-09-21T02:58:27.666160+0000","last_became_peered":"2024-09-21T02:58:27.666160+0000","last_unstale":"2024-09-21T02:59:55.771071+0000","last_undegraded":"2024-09-21T02:59:55.771071+0000","last_fullsized":"2024-09-21T02:59:55.771071+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:44:56.849666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466729+0000","last_change":"2024-09-21T02:58:25.637266+0000","last_active":"2024-09-21T02:59:55.466729+0000","last_peered":"2024-09-21T02:59:55.466729+0000","last_clean":"2024-09-21T02:59:55.466729+0000","last_became_active":"2024-09-21T02:58:25.636533+0000","last_became_peered":"2024-09-21T02:58:25.636533+0000","last_unstale":"2024-09-21T02:59:55.466729+0000","last_undegraded":"2024-09-21T02:59:55.466729+0000","last_fullsized":"2024-09-21T02:59:55.466729+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:20:22.258128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"77'9","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797346+0000","last_change":"2024-09-21T02:58:54.745298+0000","last_active":"2024-09-21T02:59:55.797346+0000","last_peered":"2024-09-21T02:59:55.797346+0000","last_clean":"2024-09-21T02:59:55.797346+0000","last_became_active":"2024-09-21T02:58:54.744775+0000","last_became_peered":"2024-09-21T02:58:54.744775+0000","last_unstale":"2024-09-21T02:59:55.797346+0000","last_undegraded":"2024-09-21T02:59:55.797346+0000","last_fullsized":"2024-09-21T02:59:55.797346+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:22:14.405920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,5],"acting":[0,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"75'3","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245494+0000","last_change":"2024-09-21T02:58:25.797791+0000","last_active":"2024-09-21T02:59:55.245494+0000","last_peered":"2024-09-21T02:59:55.245494+0000","last_clean":"2024-09-21T02:59:55.245494+0000","last_became_active":"2024-09-21T02:58:25.797274+0000","last_became_peered":"2024-09-21T02:58:25.797274+0000","last_unstale":"2024-09-21T02:59:55.245494+0000","last_undegraded":"2024-09-21T02:59:55.245494+0000","last_fullsized":"2024-09-21T02:59:55.245494+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:15:01.386953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"77'11","reported_seq":33,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769650+0000","last_change":"2024-09-21T02:58:27.656751+0000","last_active":"2024-09-21T02:59:55.769650+0000","last_peered":"2024-09-21T02:59:55.769650+0000","last_clean":"2024-09-21T02:59:55.769650+0000","last_became_active":"2024-09-21T02:58:27.655536+0000","last_became_peered":"2024-09-21T02:58:27.655536+0000","last_unstale":"2024-09-21T02:59:55.769650+0000","last_undegraded":"2024-09-21T02:59:55.769650+0000","last_fullsized":"2024-09-21T02:59:55.769650+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:11:56.197746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.14","version":"79'31","reported_seq":247,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T03:00:03.830470+0000","last_change":"2024-09-21T02:58:25.641012+0000","last_active":"2024-09-21T03:00:03.830470+0000","last_peered":"2024-09-21T03:00:03.830470+0000","last_clean":"2024-09-21T03:00:03.830470+0000","last_became_active":"2024-09-21T02:58:25.640584+0000","last_became_peered":"2024-09-21T02:58:25.640584+0000","last_unstale":"2024-09-21T03:00:03.830470+0000","last_undegraded":"2024-09-21T03:00:03.830470+0000","last_fullsized":"2024-09-21T03:00:03.830470+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":31,"log_dups_size":0,"ondisk_log_size":31,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:57:51.020271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":67514,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":189,"num_read_kb":160,"num_write":31,"num_write_kb":81,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769962+0000","last_change":"2024-09-21T02:58:27.656398+0000","last_active":"2024-09-21T02:59:55.769962+0000","last_peered":"2024-09-21T02:59:55.769962+0000","last_clean":"2024-09-21T02:59:55.769962+0000","last_became_active":"2024-09-21T02:58:27.655455+0000","last_became_peered":"2024-09-21T02:58:27.655455+0000","last_unstale":"2024-09-21T02:59:55.769962+0000","last_undegraded":"2024-09-21T02:59:55.769962+0000","last_fullsized":"2024-09-21T02:59:55.769962+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:00.901413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.13","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771792+0000","last_change":"2024-09-21T02:58:25.637265+0000","last_active":"2024-09-21T02:59:55.771792+0000","last_peered":"2024-09-21T02:59:55.771792+0000","last_clean":"2024-09-21T02:59:55.771792+0000","last_became_active":"2024-09-21T02:58:25.636797+0000","last_became_peered":"2024-09-21T02:58:25.636797+0000","last_unstale":"2024-09-21T02:59:55.771792+0000","last_undegraded":"2024-09-21T02:59:55.771792+0000","last_fullsized":"2024-09-21T02:59:55.771792+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:31:47.579793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240960+0000","last_change":"2024-09-21T02:58:27.652200+0000","last_active":"2024-09-21T02:59:55.240960+0000","last_peered":"2024-09-21T02:59:55.240960+0000","last_clean":"2024-09-21T02:59:55.240960+0000","last_became_active":"2024-09-21T02:58:27.651849+0000","last_became_peered":"2024-09-21T02:58:27.651849+0000","last_unstale":"2024-09-21T02:59:55.240960+0000","last_undegraded":"2024-09-21T02:59:55.240960+0000","last_fullsized":"2024-09-21T02:59:55.240960+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:24:59.433296+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245476+0000","last_change":"2024-09-21T02:58:25.638162+0000","last_active":"2024-09-21T02:59:55.245476+0000","last_peered":"2024-09-21T02:59:55.245476+0000","last_clean":"2024-09-21T02:59:55.245476+0000","last_became_active":"2024-09-21T02:58:25.637665+0000","last_became_peered":"2024-09-21T02:58:25.637665+0000","last_unstale":"2024-09-21T02:59:55.245476+0000","last_undegraded":"2024-09-21T02:59:55.245476+0000","last_fullsized":"2024-09-21T02:59:55.245476+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:59:38.627896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466508+0000","last_change":"2024-09-21T02:58:27.654806+0000","last_active":"2024-09-21T02:59:55.466508+0000","last_peered":"2024-09-21T02:59:55.466508+0000","last_clean":"2024-09-21T02:59:55.466508+0000","last_became_active":"2024-09-21T02:58:27.654427+0000","last_became_peered":"2024-09-21T02:58:27.654427+0000","last_unstale":"2024-09-21T02:59:55.466508+0000","last_undegraded":"2024-09-21T02:59:55.466508+0000","last_fullsized":"2024-09-21T02:59:55.466508+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:55:20.513112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242473+0000","last_change":"2024-09-21T02:58:25.634468+0000","last_active":"2024-09-21T02:59:55.242473+0000","last_peered":"2024-09-21T02:59:55.242473+0000","last_clean":"2024-09-21T02:59:55.242473+0000","last_became_active":"2024-09-21T02:58:25.634075+0000","last_became_peered":"2024-09-21T02:58:25.634075+0000","last_unstale":"2024-09-21T02:59:55.242473+0000","last_undegraded":"2024-09-21T02:59:55.242473+0000","last_fullsized":"2024-09-21T02:59:55.242473+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:00:04.213490+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466307+0000","last_change":"2024-09-21T02:58:27.662143+0000","last_active":"2024-09-21T02:59:55.466307+0000","last_peered":"2024-09-21T02:59:55.466307+0000","last_clean":"2024-09-21T02:59:55.466307+0000","last_became_active":"2024-09-21T02:58:27.661688+0000","last_became_peered":"2024-09-21T02:58:27.661688+0000","last_unstale":"2024-09-21T02:59:55.466307+0000","last_undegraded":"2024-09-21T02:59:55.466307+0000","last_fullsized":"2024-09-21T02:59:55.466307+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:14:02.994706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.10","version":"75'8","reported_seq":35,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770297+0000","last_change":"2024-09-21T02:58:25.797772+0000","last_active":"2024-09-21T02:59:55.770297+0000","last_peered":"2024-09-21T02:59:55.770297+0000","last_clean":"2024-09-21T02:59:55.770297+0000","last_became_active":"2024-09-21T02:58:25.797335+0000","last_became_peered":"2024-09-21T02:58:25.797335+0000","last_unstale":"2024-09-21T02:59:55.770297+0000","last_undegraded":"2024-09-21T02:59:55.770297+0000","last_fullsized":"2024-09-21T02:59:55.770297+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:49:15.285497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245247+0000","last_change":"2024-09-21T02:58:27.652821+0000","last_active":"2024-09-21T02:59:55.245247+0000","last_peered":"2024-09-21T02:59:55.245247+0000","last_clean":"2024-09-21T02:59:55.245247+0000","last_became_active":"2024-09-21T02:58:27.652446+0000","last_became_peered":"2024-09-21T02:58:27.652446+0000","last_unstale":"2024-09-21T02:59:55.245247+0000","last_undegraded":"2024-09-21T02:59:55.245247+0000","last_fullsized":"2024-09-21T02:59:55.245247+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:49:25.799620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771747+0000","last_change":"2024-09-21T02:58:25.638996+0000","last_active":"2024-09-21T02:59:55.771747+0000","last_peered":"2024-09-21T02:59:55.771747+0000","last_clean":"2024-09-21T02:59:55.771747+0000","last_became_active":"2024-09-21T02:58:25.637451+0000","last_became_peered":"2024-09-21T02:58:25.637451+0000","last_unstale":"2024-09-21T02:59:55.771747+0000","last_undegraded":"2024-09-21T02:59:55.771747+0000","last_fullsized":"2024-09-21T02:59:55.771747+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:51:21.267072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247441+0000","last_change":"2024-09-21T02:58:27.666963+0000","last_active":"2024-09-21T02:59:55.247441+0000","last_peered":"2024-09-21T02:59:55.247441+0000","last_clean":"2024-09-21T02:59:55.247441+0000","last_became_active":"2024-09-21T02:58:27.666290+0000","last_became_peered":"2024-09-21T02:58:27.666290+0000","last_unstale":"2024-09-21T02:59:55.247441+0000","last_undegraded":"2024-09-21T02:59:55.247441+0000","last_fullsized":"2024-09-21T02:59:55.247441+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:37:42.098949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.e","version":"75'2","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241296+0000","last_change":"2024-09-21T02:58:25.638443+0000","last_active":"2024-09-21T02:59:55.241296+0000","last_peered":"2024-09-21T02:59:55.241296+0000","last_clean":"2024-09-21T02:59:55.241296+0000","last_became_active":"2024-09-21T02:58:25.638042+0000","last_became_peered":"2024-09-21T02:58:25.638042+0000","last_unstale":"2024-09-21T02:59:55.241296+0000","last_undegraded":"2024-09-21T02:59:55.241296+0000","last_fullsized":"2024-09-21T02:59:55.241296+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:02:18.966908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466773+0000","last_change":"2024-09-21T02:58:27.659100+0000","last_active":"2024-09-21T02:59:55.466773+0000","last_peered":"2024-09-21T02:59:55.466773+0000","last_clean":"2024-09-21T02:59:55.466773+0000","last_became_active":"2024-09-21T02:58:27.658079+0000","last_became_peered":"2024-09-21T02:58:27.658079+0000","last_unstale":"2024-09-21T02:59:55.466773+0000","last_undegraded":"2024-09-21T02:59:55.466773+0000","last_fullsized":"2024-09-21T02:59:55.466773+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:22:36.723465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"75'1","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245695+0000","last_change":"2024-09-21T02:58:25.638588+0000","last_active":"2024-09-21T02:59:55.245695+0000","last_peered":"2024-09-21T02:59:55.245695+0000","last_clean":"2024-09-21T02:59:55.245695+0000","last_became_active":"2024-09-21T02:58:25.638165+0000","last_became_peered":"2024-09-21T02:58:25.638165+0000","last_unstale":"2024-09-21T02:59:55.245695+0000","last_undegraded":"2024-09-21T02:59:55.245695+0000","last_fullsized":"2024-09-21T02:59:55.245695+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:44:29.170885+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466573+0000","last_change":"2024-09-21T02:58:27.657804+0000","last_active":"2024-09-21T02:59:55.466573+0000","last_peered":"2024-09-21T02:59:55.466573+0000","last_clean":"2024-09-21T02:59:55.466573+0000","last_became_active":"2024-09-21T02:58:27.657046+0000","last_became_peered":"2024-09-21T02:58:27.657046+0000","last_unstale":"2024-09-21T02:59:55.466573+0000","last_undegraded":"2024-09-21T02:59:55.466573+0000","last_fullsized":"2024-09-21T02:59:55.466573+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:49:24.939426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.772036+0000","last_change":"2024-09-21T02:58:25.796849+0000","last_active":"2024-09-21T02:59:55.772036+0000","last_peered":"2024-09-21T02:59:55.772036+0000","last_clean":"2024-09-21T02:59:55.772036+0000","last_became_active":"2024-09-21T02:58:25.796404+0000","last_became_peered":"2024-09-21T02:58:25.796404+0000","last_unstale":"2024-09-21T02:59:55.772036+0000","last_undegraded":"2024-09-21T02:59:55.772036+0000","last_fullsized":"2024-09-21T02:59:55.772036+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:12:02.945633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466818+0000","last_change":"2024-09-21T02:58:27.656247+0000","last_active":"2024-09-21T02:59:55.466818+0000","last_peered":"2024-09-21T02:59:55.466818+0000","last_clean":"2024-09-21T02:59:55.466818+0000","last_became_active":"2024-09-21T02:58:27.655724+0000","last_became_peered":"2024-09-21T02:58:27.655724+0000","last_unstale":"2024-09-21T02:59:55.466818+0000","last_undegraded":"2024-09-21T02:59:55.466818+0000","last_fullsized":"2024-09-21T02:59:55.466818+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:37:31.360998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"22'92","reported_seq":98,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466691+0000","last_change":"2024-09-21T02:57:14.449672+0000","last_active":"2024-09-21T02:59:55.466691+0000","last_peered":"2024-09-21T02:59:55.466691+0000","last_clean":"2024-09-21T02:59:55.466691+0000","last_became_active":"2024-09-21T02:57:14.443245+0000","last_became_peered":"2024-09-21T02:57:14.443245+0000","last_unstale":"2024-09-21T02:59:55.466691+0000","last_undegraded":"2024-09-21T02:59:55.466691+0000","last_fullsized":"2024-09-21T02:59:55.466691+0000","mapping_epoch":65,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":66,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:53:55.710989+0000","last_clean_scrub_stamp":"2024-09-21T02:53:55.710989+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:27:19.225061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242571+0000","last_change":"2024-09-21T02:58:25.632939+0000","last_active":"2024-09-21T02:59:55.242571+0000","last_peered":"2024-09-21T02:59:55.242571+0000","last_clean":"2024-09-21T02:59:55.242571+0000","last_became_active":"2024-09-21T02:58:25.632007+0000","last_became_peered":"2024-09-21T02:58:25.632007+0000","last_unstale":"2024-09-21T02:59:55.242571+0000","last_undegraded":"2024-09-21T02:59:55.242571+0000","last_fullsized":"2024-09-21T02:59:55.242571+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:14:45.907546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.2","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769526+0000","last_change":"2024-09-21T02:58:27.655772+0000","last_active":"2024-09-21T02:59:55.769526+0000","last_peered":"2024-09-21T02:59:55.769526+0000","last_clean":"2024-09-21T02:59:55.769526+0000","last_became_active":"2024-09-21T02:58:27.655202+0000","last_became_peered":"2024-09-21T02:58:27.655202+0000","last_unstale":"2024-09-21T02:59:55.769526+0000","last_undegraded":"2024-09-21T02:59:55.769526+0000","last_fullsized":"2024-09-21T02:59:55.769526+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:16:58.683417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466591+0000","last_change":"2024-09-21T02:58:25.639226+0000","last_active":"2024-09-21T02:59:55.466591+0000","last_peered":"2024-09-21T02:59:55.466591+0000","last_clean":"2024-09-21T02:59:55.466591+0000","last_became_active":"2024-09-21T02:58:25.638780+0000","last_became_peered":"2024-09-21T02:58:25.638780+0000","last_unstale":"2024-09-21T02:59:55.466591+0000","last_undegraded":"2024-09-21T02:59:55.466591+0000","last_fullsized":"2024-09-21T02:59:55.466591+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:34:30.999969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.240903+0000","last_change":"2024-09-21T02:58:27.659406+0000","last_active":"2024-09-21T02:59:55.240903+0000","last_peered":"2024-09-21T02:59:55.240903+0000","last_clean":"2024-09-21T02:59:55.240903+0000","last_became_active":"2024-09-21T02:58:27.659029+0000","last_became_peered":"2024-09-21T02:58:27.659029+0000","last_unstale":"2024-09-21T02:59:55.240903+0000","last_undegraded":"2024-09-21T02:59:55.240903+0000","last_fullsized":"2024-09-21T02:59:55.240903+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T11:25:17.310633+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241097+0000","last_change":"2024-09-21T02:58:25.796063+0000","last_active":"2024-09-21T02:59:55.241097+0000","last_peered":"2024-09-21T02:59:55.241097+0000","last_clean":"2024-09-21T02:59:55.241097+0000","last_became_active":"2024-09-21T02:58:25.795615+0000","last_became_peered":"2024-09-21T02:58:25.795615+0000","last_unstale":"2024-09-21T02:59:55.241097+0000","last_undegraded":"2024-09-21T02:59:55.241097+0000","last_fullsized":"2024-09-21T02:59:55.241097+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:10:43.856797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"77'8","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770245+0000","last_change":"2024-09-21T02:58:54.909889+0000","last_active":"2024-09-21T02:59:55.770245+0000","last_peered":"2024-09-21T02:59:55.770245+0000","last_clean":"2024-09-21T02:59:55.770245+0000","last_became_active":"2024-09-21T02:58:54.909607+0000","last_became_peered":"2024-09-21T02:58:54.909607+0000","last_unstale":"2024-09-21T02:59:55.770245+0000","last_undegraded":"2024-09-21T02:59:55.770245+0000","last_fullsized":"2024-09-21T02:59:55.770245+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T08:09:48.017679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,4],"acting":[9,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466532+0000","last_change":"2024-09-21T02:58:25.634157+0000","last_active":"2024-09-21T02:59:55.466532+0000","last_peered":"2024-09-21T02:59:55.466532+0000","last_clean":"2024-09-21T02:59:55.466532+0000","last_became_active":"2024-09-21T02:58:25.633508+0000","last_became_peered":"2024-09-21T02:58:25.633508+0000","last_unstale":"2024-09-21T02:59:55.466532+0000","last_undegraded":"2024-09-21T02:59:55.466532+0000","last_fullsized":"2024-09-21T02:59:55.466532+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:02:04.440738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"77'7","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769978+0000","last_change":"2024-09-21T02:58:27.656671+0000","last_active":"2024-09-21T02:59:55.769978+0000","last_peered":"2024-09-21T02:59:55.769978+0000","last_clean":"2024-09-21T02:59:55.769978+0000","last_became_active":"2024-09-21T02:58:27.656190+0000","last_became_peered":"2024-09-21T02:58:27.656190+0000","last_unstale":"2024-09-21T02:59:55.769978+0000","last_undegraded":"2024-09-21T02:59:55.769978+0000","last_fullsized":"2024-09-21T02:59:55.769978+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T06:51:07.598340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770430+0000","last_change":"2024-09-21T02:58:25.797039+0000","last_active":"2024-09-21T02:59:55.770430+0000","last_peered":"2024-09-21T02:59:55.770430+0000","last_clean":"2024-09-21T02:59:55.770430+0000","last_became_active":"2024-09-21T02:58:25.796571+0000","last_became_peered":"2024-09-21T02:58:25.796571+0000","last_unstale":"2024-09-21T02:59:55.770430+0000","last_undegraded":"2024-09-21T02:59:55.770430+0000","last_fullsized":"2024-09-21T02:59:55.770430+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:49:54.608007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769861+0000","last_change":"2024-09-21T02:58:27.653179+0000","last_active":"2024-09-21T02:59:55.769861+0000","last_peered":"2024-09-21T02:59:55.769861+0000","last_clean":"2024-09-21T02:59:55.769861+0000","last_became_active":"2024-09-21T02:58:27.652721+0000","last_became_peered":"2024-09-21T02:58:27.652721+0000","last_unstale":"2024-09-21T02:59:55.769861+0000","last_undegraded":"2024-09-21T02:59:55.769861+0000","last_fullsized":"2024-09-21T02:59:55.769861+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:52:56.633407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771522+0000","last_change":"2024-09-21T02:58:25.796058+0000","last_active":"2024-09-21T02:59:55.771522+0000","last_peered":"2024-09-21T02:59:55.771522+0000","last_clean":"2024-09-21T02:59:55.771522+0000","last_became_active":"2024-09-21T02:58:25.795603+0000","last_became_peered":"2024-09-21T02:58:25.795603+0000","last_unstale":"2024-09-21T02:59:55.771522+0000","last_undegraded":"2024-09-21T02:59:55.771522+0000","last_fullsized":"2024-09-21T02:59:55.771522+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:08:11.092096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"77'9","reported_seq":30,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.241371+0000","last_change":"2024-09-21T02:58:27.659491+0000","last_active":"2024-09-21T02:59:55.241371+0000","last_peered":"2024-09-21T02:59:55.241371+0000","last_clean":"2024-09-21T02:59:55.241371+0000","last_became_active":"2024-09-21T02:58:27.659110+0000","last_became_peered":"2024-09-21T02:58:27.659110+0000","last_unstale":"2024-09-21T02:59:55.241371+0000","last_undegraded":"2024-09-21T02:59:55.241371+0000","last_fullsized":"2024-09-21T02:59:55.241371+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:26:42.060435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.6","version":"79'11","reported_seq":38,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770670+0000","last_change":"2024-09-21T02:58:25.637006+0000","last_active":"2024-09-21T02:59:55.770670+0000","last_peered":"2024-09-21T02:59:55.770670+0000","last_clean":"2024-09-21T02:59:55.770670+0000","last_became_active":"2024-09-21T02:58:25.636642+0000","last_became_peered":"2024-09-21T02:58:25.636642+0000","last_unstale":"2024-09-21T02:59:55.770670+0000","last_undegraded":"2024-09-21T02:59:55.770670+0000","last_fullsized":"2024-09-21T02:59:55.770670+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:16:42.335531+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"77'5","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770035+0000","last_change":"2024-09-21T02:58:27.656324+0000","last_active":"2024-09-21T02:59:55.770035+0000","last_peered":"2024-09-21T02:59:55.770035+0000","last_clean":"2024-09-21T02:59:55.770035+0000","last_became_active":"2024-09-21T02:58:27.655986+0000","last_became_peered":"2024-09-21T02:58:27.655986+0000","last_unstale":"2024-09-21T02:59:55.770035+0000","last_undegraded":"2024-09-21T02:59:55.770035+0000","last_fullsized":"2024-09-21T02:59:55.770035+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:27:42.198015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.242807+0000","last_change":"2024-09-21T02:58:25.796427+0000","last_active":"2024-09-21T02:59:55.242807+0000","last_peered":"2024-09-21T02:59:55.242807+0000","last_clean":"2024-09-21T02:59:55.242807+0000","last_became_active":"2024-09-21T02:58:25.795914+0000","last_became_peered":"2024-09-21T02:58:25.795914+0000","last_unstale":"2024-09-21T02:59:55.242807+0000","last_undegraded":"2024-09-21T02:59:55.242807+0000","last_fullsized":"2024-09-21T02:59:55.242807+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:37:20.282604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"77'7","reported_seq":36,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797033+0000","last_change":"2024-09-21T02:58:54.743546+0000","last_active":"2024-09-21T02:59:55.797033+0000","last_peered":"2024-09-21T02:59:55.797033+0000","last_clean":"2024-09-21T02:59:55.797033+0000","last_became_active":"2024-09-21T02:58:54.743107+0000","last_became_peered":"2024-09-21T02:58:54.743107+0000","last_unstale":"2024-09-21T02:59:55.797033+0000","last_undegraded":"2024-09-21T02:59:55.797033+0000","last_fullsized":"2024-09-21T02:59:55.797033+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:53:16.458721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,6],"acting":[0,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770566+0000","last_change":"2024-09-21T02:58:25.797666+0000","last_active":"2024-09-21T02:59:55.770566+0000","last_peered":"2024-09-21T02:59:55.770566+0000","last_clean":"2024-09-21T02:59:55.770566+0000","last_became_active":"2024-09-21T02:58:25.797174+0000","last_became_peered":"2024-09-21T02:58:25.797174+0000","last_unstale":"2024-09-21T02:59:55.770566+0000","last_undegraded":"2024-09-21T02:59:55.770566+0000","last_fullsized":"2024-09-21T02:59:55.770566+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:14:14.168928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"77'2","reported_seq":24,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769867+0000","last_change":"2024-09-21T02:58:27.655283+0000","last_active":"2024-09-21T02:59:55.769867+0000","last_peered":"2024-09-21T02:59:55.769867+0000","last_clean":"2024-09-21T02:59:55.769867+0000","last_became_active":"2024-09-21T02:58:27.654714+0000","last_became_peered":"2024-09-21T02:58:27.654714+0000","last_unstale":"2024-09-21T02:59:55.769867+0000","last_undegraded":"2024-09-21T02:59:55.769867+0000","last_fullsized":"2024-09-21T02:59:55.769867+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:43:38.251676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770693+0000","last_change":"2024-09-21T02:58:25.634590+0000","last_active":"2024-09-21T02:59:55.770693+0000","last_peered":"2024-09-21T02:59:55.770693+0000","last_clean":"2024-09-21T02:59:55.770693+0000","last_became_active":"2024-09-21T02:58:25.634227+0000","last_became_peered":"2024-09-21T02:58:25.634227+0000","last_unstale":"2024-09-21T02:59:55.770693+0000","last_undegraded":"2024-09-21T02:59:55.770693+0000","last_fullsized":"2024-09-21T02:59:55.770693+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:43:58.494601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247844+0000","last_change":"2024-09-21T02:58:27.661045+0000","last_active":"2024-09-21T02:59:55.247844+0000","last_peered":"2024-09-21T02:59:55.247844+0000","last_clean":"2024-09-21T02:59:55.247844+0000","last_became_active":"2024-09-21T02:58:27.660619+0000","last_became_peered":"2024-09-21T02:58:27.660619+0000","last_unstale":"2024-09-21T02:59:55.247844+0000","last_undegraded":"2024-09-21T02:59:55.247844+0000","last_fullsized":"2024-09-21T02:59:55.247844+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:12:52.612363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":26,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771806+0000","last_change":"2024-09-21T02:58:25.635393+0000","last_active":"2024-09-21T02:59:55.771806+0000","last_peered":"2024-09-21T02:59:55.771806+0000","last_clean":"2024-09-21T02:59:55.771806+0000","last_became_active":"2024-09-21T02:58:25.634840+0000","last_became_peered":"2024-09-21T02:58:25.634840+0000","last_unstale":"2024-09-21T02:59:55.771806+0000","last_undegraded":"2024-09-21T02:59:55.771806+0000","last_fullsized":"2024-09-21T02:59:55.771806+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:14:24.205864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"77'8","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.769846+0000","last_change":"2024-09-21T02:58:27.657536+0000","last_active":"2024-09-21T02:59:55.769846+0000","last_peered":"2024-09-21T02:59:55.769846+0000","last_clean":"2024-09-21T02:59:55.769846+0000","last_became_active":"2024-09-21T02:58:27.657176+0000","last_became_peered":"2024-09-21T02:58:27.657176+0000","last_unstale":"2024-09-21T02:59:55.769846+0000","last_undegraded":"2024-09-21T02:59:55.769846+0000","last_fullsized":"2024-09-21T02:59:55.769846+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T03:05:42.761917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"75'2","reported_seq":41,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771385+0000","last_change":"2024-09-21T02:58:54.914527+0000","last_active":"2024-09-21T02:59:55.771385+0000","last_peered":"2024-09-21T02:59:55.771385+0000","last_clean":"2024-09-21T02:59:55.771385+0000","last_became_active":"2024-09-21T02:58:54.910405+0000","last_became_peered":"2024-09-21T02:58:54.910405+0000","last_unstale":"2024-09-21T02:59:55.771385+0000","last_undegraded":"2024-09-21T02:59:55.771385+0000","last_fullsized":"2024-09-21T02:59:55.771385+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:40:50.419816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,7],"acting":[8,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"77'5","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.466388+0000","last_change":"2024-09-21T02:58:27.655116+0000","last_active":"2024-09-21T02:59:55.466388+0000","last_peered":"2024-09-21T02:59:55.466388+0000","last_clean":"2024-09-21T02:59:55.466388+0000","last_became_active":"2024-09-21T02:58:27.654689+0000","last_became_peered":"2024-09-21T02:58:27.654689+0000","last_unstale":"2024-09-21T02:59:55.466388+0000","last_undegraded":"2024-09-21T02:59:55.466388+0000","last_fullsized":"2024-09-21T02:59:55.466388+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:42:56.094958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1d","version":"77'15","reported_seq":37,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.245525+0000","last_change":"2024-09-21T02:58:27.655691+0000","last_active":"2024-09-21T02:59:55.245525+0000","last_peered":"2024-09-21T02:59:55.245525+0000","last_clean":"2024-09-21T02:59:55.245525+0000","last_became_active":"2024-09-21T02:58:27.655149+0000","last_became_peered":"2024-09-21T02:58:27.655149+0000","last_unstale":"2024-09-21T02:59:55.245525+0000","last_undegraded":"2024-09-21T02:59:55.245525+0000","last_fullsized":"2024-09-21T02:59:55.245525+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T14:34:16.628232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"75'2","reported_seq":29,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.771799+0000","last_change":"2024-09-21T02:58:25.642679+0000","last_active":"2024-09-21T02:59:55.771799+0000","last_peered":"2024-09-21T02:59:55.771799+0000","last_clean":"2024-09-21T02:59:55.771799+0000","last_became_active":"2024-09-21T02:58:25.642275+0000","last_became_peered":"2024-09-21T02:58:25.642275+0000","last_unstale":"2024-09-21T02:59:55.771799+0000","last_undegraded":"2024-09-21T02:59:55.771799+0000","last_fullsized":"2024-09-21T02:59:55.771799+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T12:14:22.179845+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"77'9","reported_seq":31,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770339+0000","last_change":"2024-09-21T02:58:27.666407+0000","last_active":"2024-09-21T02:59:55.770339+0000","last_peered":"2024-09-21T02:59:55.770339+0000","last_clean":"2024-09-21T02:59:55.770339+0000","last_became_active":"2024-09-21T02:58:27.665998+0000","last_became_peered":"2024-09-21T02:58:27.665998+0000","last_unstale":"2024-09-21T02:59:55.770339+0000","last_undegraded":"2024-09-21T02:59:55.770339+0000","last_fullsized":"2024-09-21T02:59:55.770339+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T09:20:56.095316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1d","version":"75'2","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.247332+0000","last_change":"2024-09-21T02:58:54.918104+0000","last_active":"2024-09-21T02:59:55.247332+0000","last_peered":"2024-09-21T02:59:55.247332+0000","last_clean":"2024-09-21T02:59:55.247332+0000","last_became_active":"2024-09-21T02:58:54.911376+0000","last_became_peered":"2024-09-21T02:58:54.911376+0000","last_unstale":"2024-09-21T02:59:55.247332+0000","last_undegraded":"2024-09-21T02:59:55.247332+0000","last_fullsized":"2024-09-21T02:59:55.247332+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T05:21:42.992181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,7],"acting":[11,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.1f","version":"77'6","reported_seq":28,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.797210+0000","last_change":"2024-09-21T02:58:27.663774+0000","last_active":"2024-09-21T02:59:55.797210+0000","last_peered":"2024-09-21T02:59:55.797210+0000","last_clean":"2024-09-21T02:59:55.797210+0000","last_became_active":"2024-09-21T02:58:27.663311+0000","last_became_peered":"2024-09-21T02:58:27.663311+0000","last_unstale":"2024-09-21T02:59:55.797210+0000","last_undegraded":"2024-09-21T02:59:55.797210+0000","last_fullsized":"2024-09-21T02:59:55.797210+0000","mapping_epoch":73,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":74,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T13:00:26.782986+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"75'1","reported_seq":27,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770242+0000","last_change":"2024-09-21T02:58:25.796033+0000","last_active":"2024-09-21T02:59:55.770242+0000","last_peered":"2024-09-21T02:59:55.770242+0000","last_clean":"2024-09-21T02:59:55.770242+0000","last_became_active":"2024-09-21T02:58:25.795583+0000","last_became_peered":"2024-09-21T02:58:25.795583+0000","last_unstale":"2024-09-21T02:59:55.770242+0000","last_undegraded":"2024-09-21T02:59:55.770242+0000","last_fullsized":"2024-09-21T02:59:55.770242+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T04:28:55.428532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"77'5","reported_seq":21,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770500+0000","last_change":"2024-09-21T02:58:54.743356+0000","last_active":"2024-09-21T02:59:55.770500+0000","last_peered":"2024-09-21T02:59:55.770500+0000","last_clean":"2024-09-21T02:59:55.770500+0000","last_became_active":"2024-09-21T02:58:54.742875+0000","last_became_peered":"2024-09-21T02:58:54.742875+0000","last_unstale":"2024-09-21T02:59:55.770500+0000","last_undegraded":"2024-09-21T02:59:55.770500+0000","last_fullsized":"2024-09-21T02:59:55.770500+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":73,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:26.622661+0000","last_clean_scrub_stamp":"2024-09-21T02:58:26.622661+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T10:29:47.202904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,5],"acting":[9,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.1f","version":"75'4","reported_seq":32,"reported_epoch":79,"state":"active+clean","last_fresh":"2024-09-21T02:59:55.770077+0000","last_change":"2024-09-21T02:58:25.635375+0000","last_active":"2024-09-21T02:59:55.770077+0000","last_peered":"2024-09-21T02:59:55.770077+0000","last_clean":"2024-09-21T02:59:55.770077+0000","last_became_active":"2024-09-21T02:58:25.635007+0000","last_became_peered":"2024-09-21T02:58:25.635007+0000","last_unstale":"2024-09-21T02:59:55.770077+0000","last_undegraded":"2024-09-21T02:59:55.770077+0000","last_fullsized":"2024-09-21T02:59:55.770077+0000","mapping_epoch":71,"log_start":"0'0","ondisk_log_start":"0'0","created":71,"last_epoch_clean":72,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-09-21T02:58:24.609023+0000","last_clean_scrub_stamp":"2024-09-21T02:58:24.609023+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-09-22T07:34:23.084716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":702,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":502,"ondisk_log_size":502,"up":192,"acting":192,"num_store_stats":12},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":68312,"num_objects":22,"num_object_clones":0,"num_object_copies":66,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":195,"num_read_kb":166,"num_write":91,"num_write_kb":98,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":204936,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":74,"ondisk_log_size":74,"up":96,"acting":96,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":69,"num_write":123,"num_write_kb":2150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":10,"num_bytes_recovered":2951840,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":92,"ondisk_log_size":92,"up":3,"acting":3,"num_store_stats":8}],"osd_stats":[{"osd":11,"up_from":69,"seq":296352743457,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":848,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":868352,"data_stored":430015,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":27654596},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":64,"seq":274877906982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":41028,"kb_used_data":1520,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":93704124,"statfs":{"total":95995035648,"available":95953022976,"internally_reserved":0,"allocated":1556480,"data_stored":1087900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":40434112},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":59,"seq":253403070507,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27880,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717272,"statfs":{"total":95995035648,"available":95966486528,"internally_reserved":0,"allocated":872448,"data_stored":430597,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,7,8,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":54,"seq":231928234032,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":856,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":876544,"data_stored":430619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":48,"seq":206158430260,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36788,"kb_used_data":1436,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708364,"statfs":{"total":95995035648,"available":95957364736,"internally_reserved":0,"allocated":1470464,"data_stored":1021082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":36174282},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":43,"seq":184683593786,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36200,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708952,"statfs":{"total":95995035648,"available":95957966848,"internally_reserved":0,"allocated":872448,"data_stored":430132,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,2,3,4,5,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757310,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36796,"kb_used_data":1444,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708356,"statfs":{"total":95995035648,"available":95957356544,"internally_reserved":0,"allocated":1478656,"data_stored":1020682,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":32,"seq":137438953539,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36204,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708948,"statfs":{"total":95995035648,"available":95957962752,"internally_reserved":0,"allocated":872448,"data_stored":430037,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":26,"seq":111669149768,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27892,"kb_used_data":864,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717260,"statfs":{"total":95995035648,"available":95966474240,"internally_reserved":0,"allocated":884736,"data_stored":430353,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":19,"seq":81604378703,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36276,"kb_used_data":928,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708876,"statfs":{"total":95995035648,"available":95957889024,"internally_reserved":0,"allocated":950272,"data_stored":497563,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":14,"seq":60129542226,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27956,"kb_used_data":924,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717196,"statfs":{"total":95995035648,"available":95966408704,"internally_reserved":0,"allocated":946176,"data_stored":497221,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705752,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27884,"kb_used_data":852,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717268,"statfs":{"total":95995035648,"available":95966482432,"internally_reserved":0,"allocated":872448,"data_stored":430105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":77824,"data_stored":67514,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":81920,"data_stored":67548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":90112,"data_stored":67708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":234,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":117,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-09-21T03:00:07.987 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:08.279 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:08.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:08 smithi016 ceph-mon[31253]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 943 B/s rd, 171 B/s wr, 2 op/s 2024-09-21T03:00:08.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:08 smithi016 ceph-mon[31253]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.559 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:08 smithi016 ceph-mon[31253]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:08 smithi018 ceph-mon[34587]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 943 B/s rd, 171 B/s wr, 2 op/s 2024-09-21T03:00:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:08 smithi018 ceph-mon[34587]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:08 smithi018 ceph-mon[34587]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:08 smithi123 ceph-mon[34645]: pgmap v314: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 943 B/s rd, 171 B/s wr, 2 op/s 2024-09-21T03:00:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:08 smithi123 ceph-mon[34645]: from='client.15339 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:08 smithi123 ceph-mon[34645]: from='client.15345 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 -- 172.21.15.16:0/3727122044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30109df0 msgr2=0x7fde30110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 --2- 172.21.15.16:0/3727122044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30109df0 0x7fde30110620 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fde28009ca0 tx=0x7fde2802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 -- 172.21.15.16:0/3727122044 shutdown_connections 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 --2- 172.21.15.16:0/3727122044 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30109df0 0x7fde30110620 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 --2- 172.21.15.16:0/3727122044 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde301098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 --2- 172.21.15.16:0/3727122044 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30103b60 0x7fde30103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 -- 172.21.15.16:0/3727122044 >> 172.21.15.16:0/3727122044 conn(0x7fde3007b850 msgr2=0x7fde3007bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 -- 172.21.15.16:0/3727122044 shutdown_connections 2024-09-21T03:00:08.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.784+0000 7fde35a26640 1 -- 172.21.15.16:0/3727122044 wait complete. 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 Processor -- start 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 -- start start 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde3019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 0x7fde3019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde30113ab0 con 0x7fde30103b60 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde30113930 con 0x7fde30109df0 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde35a26640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde30113c30 con 0x7fde301094a0 2024-09-21T03:00:08.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde2effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49628/0 (socket says 172.21.15.16:49628) 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.785+0000 7fde2effd640 1 -- 172.21.15.16:0/2911132317 learned_addr learned my addr 172.21.15.16:0/2911132317 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2f7fe640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 0x7fde3019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde27fff640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde3019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 -- 172.21.15.16:0/2911132317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 msgr2=0x7fde3019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde3019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 -- 172.21.15.16:0/2911132317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 msgr2=0x7fde3019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 0x7fde3019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 -- 172.21.15.16:0/2911132317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde301a03d0 con 0x7fde30103b60 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde27fff640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde3019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2f7fe640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 0x7fde3019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde2effd640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d950 tx=0x7fde2000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde20017030 con 0x7fde30103b60 2024-09-21T03:00:08.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde20004020 con 0x7fde30103b60 2024-09-21T03:00:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde200047a0 con 0x7fde30103b60 2024-09-21T03:00:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde301a06c0 con 0x7fde30103b60 2024-09-21T03:00:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.786+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde301a6e20 con 0x7fde30103b60 2024-09-21T03:00:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.787+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde30103f70 con 0x7fde30103b60 2024-09-21T03:00:08.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.789+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fde20009a30 con 0x7fde30103b60 2024-09-21T03:00:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.790+0000 7fde34a24640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 0x7fde1c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.790+0000 7fde27fff640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 0x7fde1c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:08.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.790+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fde200e76c0 con 0x7fde30103b60 2024-09-21T03:00:08.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.790+0000 7fde27fff640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 0x7fde1c0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde18005fd0 tx=0x7fde180094d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:08.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:08.793+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fde200addf0 con 0x7fde30103b60 2024-09-21T03:00:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.020+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fde3019b590 con 0x7fde30103b60 2024-09-21T03:00:09.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.021+0000 7fde34a24640 1 -- 172.21.15.16:0/2911132317 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7fde200b21d0 con 0x7fde30103b60 2024-09-21T03:00:09.023 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:09.023 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:00:09.023 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 msgr2=0x7fde1c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 0x7fde1c0b8a70 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fde18005fd0 tx=0x7fde180094d0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 msgr2=0x7fde3019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fde2000d950 tx=0x7fde2000de10 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 shutdown_connections 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde1c0b6560 0x7fde1c0b8a70 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde30109df0 0x7fde3019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde301094a0 0x7fde3019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 --2- 172.21.15.16:0/2911132317 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde30103b60 0x7fde3019a460 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:09.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 >> 172.21.15.16:0/2911132317 conn(0x7fde3007b850 msgr2=0x7fde301074f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 shutdown_connections 2024-09-21T03:00:09.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:09.023+0000 7fde35a26640 1 -- 172.21.15.16:0/2911132317 wait complete. 2024-09-21T03:00:09.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:09 smithi016 ceph-mon[31253]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:09.543 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:09 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2911132317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:09.621 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:09 smithi123 ceph-mon[34645]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:09.622 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:09 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2911132317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:09 smithi018 ceph-mon[34587]: from='client.15351 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-09-21T03:00:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:09 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2911132317' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:09.796 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:00:10.059 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:10.460 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:10 smithi016 ceph-mon[31253]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 -- 172.21.15.16:0/1269219514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568109e40 msgr2=0x7f4568110670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1269219514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568109e40 0x7f4568110670 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f4560009ca0 tx=0x7f456002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 -- 172.21.15.16:0/1269219514 shutdown_connections 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1269219514 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568109e40 0x7f4568110670 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1269219514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f45681094f0 0x7f4568109900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1269219514 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4568103bb0 0x7f4568103fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 -- 172.21.15.16:0/1269219514 >> 172.21.15.16:0/1269219514 conn(0x7f45680ff600 msgr2=0x7f4568101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 -- 172.21.15.16:0/1269219514 shutdown_connections 2024-09-21T03:00:10.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.552+0000 7f456f3e0640 1 -- 172.21.15.16:0/1269219514 wait complete. 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 Processor -- start 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 -- start start 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45681094f0 0x7f456819cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 0x7f45681a1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4568113b00 con 0x7f4568103bb0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4568113980 con 0x7f45681094f0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456f3e0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4568113c80 con 0x7f4568109e40 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456d17c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456d17c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49644/0 (socket says 172.21.15.16:49644) 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.553+0000 7f456d17c640 1 -- 172.21.15.16:0/1074890616 learned_addr learned my addr 172.21.15.16:0/1074890616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456c97b640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45681094f0 0x7f456819cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d97d640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 0x7f45681a1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 -- 172.21.15.16:0/1074890616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 msgr2=0x7f45681a1ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 0x7f45681a1ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 -- 172.21.15.16:0/1074890616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45681094f0 msgr2=0x7f456819cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45681094f0 0x7f456819cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 -- 172.21.15.16:0/1074890616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45681a2600 con 0x7f4568103bb0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d97d640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 0x7f45681a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456d17c640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f456400cd80 con 0x7f4568103bb0 2024-09-21T03:00:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45681a28f0 con 0x7f4568103bb0 2024-09-21T03:00:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45681a9050 con 0x7f4568103bb0 2024-09-21T03:00:10.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45640108e0 con 0x7f4568103bb0 2024-09-21T03:00:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.554+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4564018980 con 0x7f4568103bb0 2024-09-21T03:00:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.555+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4568103fc0 con 0x7f4568103bb0 2024-09-21T03:00:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.557+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f4564010430 con 0x7f4568103bb0 2024-09-21T03:00:10.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.558+0000 7f455e7fc640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 0x7f45540b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.558+0000 7f456c97b640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 0x7f45540b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.558+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f45640e7d60 con 0x7f4568103bb0 2024-09-21T03:00:10.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.559+0000 7f456c97b640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 0x7f45540b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4568105ba0 tx=0x7f4558005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:10.562 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.561+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f45640ae490 con 0x7f4568103bb0 2024-09-21T03:00:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:10 smithi018 ceph-mon[34587]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:00:10.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:10 smithi123 ceph-mon[34645]: pgmap v315: 97 pgs: 97 active+clean; 643 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 170 B/s wr, 1 op/s 2024-09-21T03:00:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.749+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f456819d7c0 con 0x7f4568103bb0 2024-09-21T03:00:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.751+0000 7f455e7fc640 1 -- 172.21.15.16:0/1074890616 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) ==== 74+0+23127 (secure 0 0 0) 0x7f45640b2870 con 0x7f4568103bb0 2024-09-21T03:00:10.752 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:10.752 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":79,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T02:59:55.231626+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 msgr2=0x7f45540b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 0x7f45540b8a70 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f4568105ba0 tx=0x7f4558005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 msgr2=0x7f456819c690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f456400e9f0 tx=0x7f456400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 shutdown_connections 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f45540b6560 0x7f45540b8a70 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4568109e40 0x7f45681a1ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f45681094f0 0x7f456819cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 --2- 172.21.15.16:0/1074890616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4568103bb0 0x7f456819c690 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 >> 172.21.15.16:0/1074890616 conn(0x7f45680ff600 msgr2=0x7f4568108b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 shutdown_connections 2024-09-21T03:00:10.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:10.753+0000 7f456f3e0640 1 -- 172.21.15.16:0/1074890616 wait complete. 2024-09-21T03:00:11.397 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:11 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1074890616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:11.429 DEBUG:tasks.fwd_scrub:Yielding 2024-09-21T03:00:11.429 INFO:teuthology.run_tasks:Running task exec... 2024-09-21T03:00:11.440 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:00:11.441 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:00:11.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-09-21T03:00:11.441 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-09-21T03:00:11.441 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:00:11.441 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:00:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- 172.21.15.16:0/1486619183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 msgr2=0x7f987c1086f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 --2- 172.21.15.16:0/1486619183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c1086f0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9864009900 tx=0x7f986402a410 comp rx=0 tx=0).stop 2024-09-21T03:00:11.563 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- 172.21.15.16:0/1486619183 shutdown_connections 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 --2- 172.21.15.16:0/1486619183 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c1086f0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 --2- 172.21.15.16:0/1486619183 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c108c30 0x7f987c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 --2- 172.21.15.16:0/1486619183 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c1029b0 0x7f987c102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- 172.21.15.16:0/1486619183 >> 172.21.15.16:0/1486619183 conn(0x7f987c07a740 msgr2=0x7f987c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- 172.21.15.16:0/1486619183 shutdown_connections 2024-09-21T03:00:11.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- 172.21.15.16:0/1486619183 wait complete. 2024-09-21T03:00:11.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 Processor -- start 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.562+0000 7f98829e6640 1 -- start start 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 0x7f987c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c108c30 0x7f987c19fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f987c1128f0 con 0x7f987c1082e0 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f987c112770 con 0x7f987c1029b0 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f98829e6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f987c112a70 con 0x7f987c108c30 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49668/0 (socket says 172.21.15.16:49668) 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 -- 172.21.15.16:0/2664643707 learned_addr learned my addr 172.21.15.16:0/2664643707 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f987bfff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 0x7f987c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:11.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9880f83640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c108c30 0x7f987c19fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 -- 172.21.15.16:0/2664643707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c108c30 msgr2=0x7f987c19fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c108c30 0x7f987c19fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 -- 172.21.15.16:0/2664643707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 msgr2=0x7f987c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 0x7f987c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 -- 172.21.15.16:0/2664643707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f987c1a03d0 con 0x7f987c1082e0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f987bfff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 0x7f987c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9873fff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f986402a930 tx=0x7f9864004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.563+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f986403f030 con 0x7f987c1082e0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.564+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f987c1a0660 con 0x7f987c1082e0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.564+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9864003aa0 con 0x7f987c1082e0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.564+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9864035070 con 0x7f987c1082e0 2024-09-21T03:00:11.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.564+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f987c1a6e50 con 0x7f987c1082e0 2024-09-21T03:00:11.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.565+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f987c103c20 con 0x7f987c1082e0 2024-09-21T03:00:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.567+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f9864042030 con 0x7f987c1082e0 2024-09-21T03:00:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.568+0000 7f9879ffb640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 0x7f98680b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:11.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.568+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9864107920 con 0x7f987c1082e0 2024-09-21T03:00:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.568+0000 7f987bfff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 0x7f98680b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:11.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.569+0000 7f987bfff640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 0x7f98680b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f986c009b40 tx=0x7f986c005e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:11.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.573+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f98640ce050 con 0x7f987c1082e0 2024-09-21T03:00:11.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:11 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1074890616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:11.712 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:11.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:11 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1074890616' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:00:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.794+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7f987c101b50 con 0x7f98680b6560 2024-09-21T03:00:11.957 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.956+0000 7f9879ffb640 1 -- 172.21.15.16:0/2664643707 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f987c101b50 con 0x7f98680b6560 2024-09-21T03:00:11.957 INFO:teuthology.orchestra.run.smithi016.stdout:Schedule set for path / 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 msgr2=0x7f98680b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 0x7f98680b8a70 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f986c009b40 tx=0x7f986c005e50 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 msgr2=0x7f987c19aa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f986402a930 tx=0x7f9864004270 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 shutdown_connections 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98680b6560 0x7f98680b8a70 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f987c1082e0 0x7f987c19aa40 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f987c108c30 0x7f987c19fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 --2- 172.21.15.16:0/2664643707 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f987c1029b0 0x7f987c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 >> 172.21.15.16:0/2664643707 conn(0x7f987c07a740 msgr2=0x7f987c101440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 shutdown_connections 2024-09-21T03:00:11.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:11.958+0000 7f98829e6640 1 -- 172.21.15.16:0/2664643707 wait complete. 2024-09-21T03:00:11.971 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 -- 172.21.15.16:0/1569943069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1029b0 msgr2=0x7f895c102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 --2- 172.21.15.16:0/1569943069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1029b0 0x7f895c102dc0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f8958009900 tx=0x7f895802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 -- 172.21.15.16:0/1569943069 shutdown_connections 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 --2- 172.21.15.16:0/1569943069 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c108c30 0x7f895c10f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 --2- 172.21.15.16:0/1569943069 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 0x7f895c1086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 --2- 172.21.15.16:0/1569943069 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c1029b0 0x7f895c102dc0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 -- 172.21.15.16:0/1569943069 >> 172.21.15.16:0/1569943069 conn(0x7f895c07a740 msgr2=0x7f895c07ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 -- 172.21.15.16:0/1569943069 shutdown_connections 2024-09-21T03:00:12.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.089+0000 7f896393a640 1 -- 172.21.15.16:0/1569943069 wait complete. 2024-09-21T03:00:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 Processor -- start 2024-09-21T03:00:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 -- start start 2024-09-21T03:00:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c1029b0 0x7f895c19b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 0x7f895c19b7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f895c1128f0 con 0x7f895c108c30 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f895c112770 con 0x7f895c1082e0 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f896393a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f895c112a70 con 0x7f895c1029b0 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f8961ed7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f8961ed7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49700/0 (socket says 172.21.15.16:49700) 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f89616d6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c1029b0 0x7f895c19b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f8961ed7640 1 -- 172.21.15.16:0/1983310808 learned_addr learned my addr 172.21.15.16:0/1983310808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.090+0000 7f8960ed5640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 0x7f895c19b7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 -- 172.21.15.16:0/1983310808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c1029b0 msgr2=0x7f895c19b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c1029b0 0x7f895c19b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 -- 172.21.15.16:0/1983310808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 msgr2=0x7f895c19b7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 0x7f895c19b7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 -- 172.21.15.16:0/1983310808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f895c1a64f0 con 0x7f895c108c30 2024-09-21T03:00:12.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f8961ed7640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8954009fd0 tx=0x7f895400b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954017070 con 0x7f895c108c30 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f895c1a67e0 con 0x7f895c108c30 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8954003e20 con 0x7f895c108c30 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8954010460 con 0x7f895c108c30 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.091+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f895c1a6d30 con 0x7f895c108c30 2024-09-21T03:00:12.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.092+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f895c102dc0 con 0x7f895c108c30 2024-09-21T03:00:12.096 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.095+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f8954003970 con 0x7f895c108c30 2024-09-21T03:00:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.095+0000 7f89527fc640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 0x7f89480b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.096+0000 7f89616d6640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 0x7f89480b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.096+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f89540e74f0 con 0x7f895c108c30 2024-09-21T03:00:12.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.096+0000 7f89616d6640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 0x7f89480b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f8958031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.099+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f89540b2060 con 0x7f895c108c30 2024-09-21T03:00:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 -- 172.21.15.16:0/602795450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c54108cf0 msgr2=0x7f2c5410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 --2- 172.21.15.16:0/602795450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c54108cf0 0x7f2c5410f520 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c009ca0 tx=0x7f2c4c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 -- 172.21.15.16:0/602795450 shutdown_connections 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 --2- 172.21.15.16:0/602795450 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c54108cf0 0x7f2c5410f520 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 --2- 172.21.15.16:0/602795450 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c541083a0 0x7f2c541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 --2- 172.21.15.16:0/602795450 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54102a60 0x7f2c54102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 -- 172.21.15.16:0/602795450 >> 172.21.15.16:0/602795450 conn(0x7f2c54079730 msgr2=0x7f2c54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 -- 172.21.15.16:0/602795450 shutdown_connections 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.264+0000 7f2c58e56640 1 -- 172.21.15.16:0/602795450 wait complete. 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 Processor -- start 2024-09-21T03:00:12.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 -- start start 2024-09-21T03:00:12.268 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c54102a60 0x7f2c540ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 0x7f2c541000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54112960 con 0x7f2c541083a0 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541127e0 con 0x7f2c54102a60 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.265+0000 7f2c58e56640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c54112ae0 con 0x7f2c54108cf0 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49716/0 (socket says 172.21.15.16:49716) 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c5259c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c54102a60 0x7f2c540ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 -- 172.21.15.16:0/1463119500 learned_addr learned my addr 172.21.15.16:0/1463119500 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 0x7f2c541000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 -- 172.21.15.16:0/1463119500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 msgr2=0x7f2c541000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 0x7f2c541000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 -- 172.21.15.16:0/1463119500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c54102a60 msgr2=0x7f2c540ff670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c54102a60 0x7f2c540ff670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.269 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 -- 172.21.15.16:0/1463119500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c541a8760 con 0x7f2c541083a0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 0x7f2c541000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800d770 tx=0x7f2c4800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c437fe640 1 -- 172.21.15.16:0/1463119500 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48017070 con 0x7f2c541083a0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c541a8a50 con 0x7f2c541083a0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c437fe640 1 -- 172.21.15.16:0/1463119500 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c48009c30 con 0x7f2c541083a0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.266+0000 7f2c437fe640 1 -- 172.21.15.16:0/1463119500 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4800f620 con 0x7f2c541083a0 2024-09-21T03:00:12.270 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.267+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c541a8fe0 con 0x7f2c541083a0 2024-09-21T03:00:12.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 Processor -- start 2024-09-21T03:00:12.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 -- start start 2024-09-21T03:00:12.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d50b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d50beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c99d481e60 con 0x55c99d164350 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c99d471290 con 0x55c99d13b490 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c99d4750a0 con 0x55c99d143670 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49730/0 (socket says 172.21.15.16:49730) 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c52d9d640 1 -- 172.21.15.16:0/140285221 learned_addr learned my addr 172.21.15.16:0/140285221 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.270+0000 7f2c5259c640 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d50b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 -- 172.21.15.16:0/140285221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 msgr2=0x55c99d50beb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d50beb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 -- 172.21.15.16:0/140285221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 msgr2=0x55c99d50b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d50b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 -- 172.21.15.16:0/140285221 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99d1cf930 con 0x55c99d164350 2024-09-21T03:00:12.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c437fe640 1 -- 172.21.15.16:0/1463119500 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f2c4801a070 con 0x7f2c541083a0 2024-09-21T03:00:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c033040 tx=0x7f2c4c02f3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1312fc943af643a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c40ff9640 1 -- 172.21.15.16:0/140285221 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c031650 con 0x55c99d164350 2024-09-21T03:00:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c40ff9640 1 -- 172.21.15.16:0/140285221 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c4c036040 con 0x55c99d164350 2024-09-21T03:00:12.274 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.271+0000 7f2c40ff9640 1 -- 172.21.15.16:0/140285221 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c004540 con 0x55c99d164350 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c437fe640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 0x7f2c3c0b8a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5259c640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 0x7f2c3c0b8a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c437fe640 1 -- 172.21.15.16:0/1463119500 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2c480e6e80 con 0x7f2c541083a0 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/140285221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 msgr2=0x55c99d50b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c033040 tx=0x7f2c4c02f3c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5259c640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 0x7f2c3c0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c99d50a0b0 tx=0x7f2c44005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/140285221 shutdown_connections 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d50beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d50b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/140285221 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d50b430 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.272+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/140285221 >> 172.21.15.16:0/140285221 conn(0x55c99d158740 msgr2=0x55c99d509320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/140285221 shutdown_connections 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/140285221 wait complete. 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 -- start start 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 Processor -- start 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d5170c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d517660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.273+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c99d2d2890 con 0x55c99d164350 2024-09-21T03:00:12.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c99d40bd40 con 0x55c99d13b490 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c99d519a30 con 0x55c99d143670 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5f3ca740 10 client.0 _ll_register_callbacks cb 0x55c99d46fc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49734/0 (socket says 172.21.15.16:49734) 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 -- 172.21.15.16:0/393385298 learned_addr learned my addr 172.21.15.16:0/393385298 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5259c640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d5170c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d517660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 -- 172.21.15.16:0/393385298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 msgr2=0x55c99d517660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d517660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 -- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 msgr2=0x55c99d5170c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d5170c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 -- 172.21.15.16:0/393385298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99d523360 con 0x55c99d164350 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d517660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c5259c640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d5170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:12.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02f500 tx=0x7f2c4c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a51431118cf4da14 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.274+0000 7f2c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c004260 con 0x55c99d164350 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2c4c004400 con 0x55c99d164350 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c4c036040 con 0x55c99d164350 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2c4c05e050 con 0x55c99d164350 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c5f3ca740 10 client.15393 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:12.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c99d525c90 con 0x55c99d164350 2024-09-21T03:00:12.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.275+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c4c036040 con 0x55c99d164350 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2c4c04f070 con 0x55c99d164350 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 10 client.15393 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 10 client.15393 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 20 client.15393 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d122cd0 0x55c99d528ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 4 client.15393 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c99d483db0 con 0x55c99d122cd0 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.276+0000 7f2c5259c640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d122cd0 0x55c99d528ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.277+0000 7f2c39ffb640 10 client.15393 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.277+0000 7f2c5259c640 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d122cd0 0x55c99d528ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f6d9932eb152cb2 server_cookie=c416f92ff02af181 in_seq=0 out_seq=0 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.277+0000 7f2c39ffb640 1 -- 172.21.15.16:0/393385298 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c99d483db0 con 0x55c99d122cd0 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.277+0000 7f2c39ffb640 10 client.15393 handle_command_reply: tid=0 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.277+0000 7f2c5f3ca740 1 client.15393 shutdown 2024-09-21T03:00:12.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 msgr2=0x55c99d522d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c02f500 tx=0x7f2c4c003c00 comp rx=0 tx=0).stop 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 shutdown_connections 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d122cd0 0x55c99d528ff0 crc :-1 s=READY pgs=23 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d164350 0x55c99d522d20 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d517660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/393385298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d5170c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c39ffb640 0 client.15393 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 >> 172.21.15.16:0/393385298 conn(0x55c99d158740 msgr2=0x55c99d50e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 shutdown_connections 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/393385298 wait complete. 2024-09-21T03:00:12.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.278+0000 7f2c5f3ca740 20 client.15393 trim_cache size 0 max 16384 2024-09-21T03:00:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 Processor -- start 2024-09-21T03:00:12.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 -- start start 2024-09-21T03:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 0x55c99d522c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d510bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541af910 con 0x55c99d13b490 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541b2190 con 0x55c99d122cd0 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541aff80 con 0x55c99d143670 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49738/0 (socket says 172.21.15.16:49738) 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 -- 172.21.15.16:0/2858018377 learned_addr learned my addr 172.21.15.16:0/2858018377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d510bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5259c640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 0x55c99d522c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 -- 172.21.15.16:0/2858018377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 msgr2=0x55c99d510bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d510bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 -- 172.21.15.16:0/2858018377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 msgr2=0x55c99d522c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 0x55c99d522c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c51d9b640 1 -- 172.21.15.16:0/2858018377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c99d512750 con 0x55c99d13b490 2024-09-21T03:00:12.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c5259c640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 0x55c99d522c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.297+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d510bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.298+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c48000c00 tx=0x7f2c48010c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d79c9834752a4a54 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.298+0000 7f2c3bfff640 1 -- 172.21.15.16:0/2858018377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c48021070 con 0x55c99d13b490 2024-09-21T03:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.298+0000 7f2c3bfff640 1 -- 172.21.15.16:0/2858018377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c480acad0 con 0x55c99d13b490 2024-09-21T03:00:12.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.298+0000 7f2c3bfff640 1 -- 172.21.15.16:0/2858018377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c480abef0 con 0x55c99d13b490 2024-09-21T03:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/2858018377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 msgr2=0x55c99d5106b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f2c48000c00 tx=0x7f2c48010c00 comp rx=0 tx=0).stop 2024-09-21T03:00:12.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/2858018377 shutdown_connections 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d13b490 0x55c99d5106b0 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d143670 0x55c99d510bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/2858018377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d122cd0 0x55c99d522c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/2858018377 >> 172.21.15.16:0/2858018377 conn(0x55c99d158740 msgr2=0x55c99d515740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/2858018377 shutdown_connections 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/2858018377 wait complete. 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 -- start start 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.299+0000 7f2c5f3ca740 1 Processor -- start 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d122cd0 0x55c99d51ec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d51f1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541b0650 con 0x55c99d143670 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541b1ac0 con 0x55c99d13b490 2024-09-21T03:00:12.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2c541b0d20 con 0x55c99d122cd0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5f3ca740 10 client.0 _ll_register_callbacks cb 0x7f2c4c02f310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49750/0 (socket says 172.21.15.16:49750) 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 -- 172.21.15.16:0/699209894 learned_addr learned my addr 172.21.15.16:0/699209894 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c51d9b640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d51f1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c5259c640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d122cd0 0x55c99d51ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 -- 172.21.15.16:0/699209894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d122cd0 msgr2=0x55c99d51ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d122cd0 0x55c99d51ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 -- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 msgr2=0x55c99d51f1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d51f1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 -- 172.21.15.16:0/699209894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c99d5179d0 con 0x55c99d143670 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c52d9d640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c031990 tx=0x7f2c4c0319c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c269667f21edf59e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.300+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2c4c02fec0 con 0x55c99d143670 2024-09-21T03:00:12.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2c4c0305c0 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2c480acad0 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2c4c032eb0 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 10 client.15399 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c99d593930 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2c480acad0 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2c4c06a070 con 0x55c99d143670 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 10 client.15399 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 10 client.15399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 20 client.15399 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d164350 0x55c99d5960e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 4 client.15399 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.301+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55c99d47f860 con 0x55c99d164350 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.302+0000 7f2c5259c640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d164350 0x55c99d5960e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.302+0000 7f2c39ffb640 10 client.15399 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:12.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.302+0000 7f2c5259c640 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d164350 0x55c99d5960e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=af3f0d3b6c0206a6 server_cookie=3ff676779bde6b99 in_seq=0 out_seq=0 2024-09-21T03:00:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.304+0000 7f2c39ffb640 1 -- 172.21.15.16:0/699209894 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55c99d47f860 con 0x55c99d164350 2024-09-21T03:00:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.304+0000 7f2c39ffb640 10 client.15399 handle_command_reply: tid=0 2024-09-21T03:00:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.304+0000 7f2c5f3ca740 1 client.15399 shutdown 2024-09-21T03:00:12.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 msgr2=0x55c99d519c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f2c4c031990 tx=0x7f2c4c0319c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 shutdown_connections 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c99d164350 0x55c99d5960e0 crc :-1 s=READY pgs=24 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c99d143670 0x55c99d519c40 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c99d13b490 0x55c99d51f1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 --2- 172.21.15.16:0/699209894 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c99d122cd0 0x55c99d51ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 >> 172.21.15.16:0/699209894 conn(0x55c99d158740 msgr2=0x55c99d50e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c39ffb640 0 client.15399 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 shutdown_connections 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 1 -- 172.21.15.16:0/699209894 wait complete. 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c5f3ca740 20 client.15399 trim_cache size 0 max 16384 2024-09-21T03:00:12.308 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4f883922-f7c1-4b58-a99c-95533f769ca1", 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 msgr2=0x7f2c3c0b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.305+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 0x7f2c3c0b8a70 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55c99d50a0b0 tx=0x7f2c44005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 msgr2=0x7f2c540ffbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f2c4800d770 tx=0x7f2c4800dc30 comp rx=0 tx=0).stop 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 shutdown_connections 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2c3c0b6560 0x7f2c3c0b8a70 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2c541083a0 0x7f2c540ffbb0 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c54108cf0 0x7f2c541000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 --2- 172.21.15.16:0/1463119500 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2c54102a60 0x7f2c540ff670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 >> 172.21.15.16:0/1463119500 conn(0x7f2c54079730 msgr2=0x7f2c541019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 shutdown_connections 2024-09-21T03:00:12.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.306+0000 7f2c58e56640 1 -- 172.21.15.16:0/1463119500 wait complete. 2024-09-21T03:00:12.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.321+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f895c0638a0 con 0x7f89480b6560 2024-09-21T03:00:12.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.386+0000 7f89527fc640 1 -- 172.21.15.16:0/1983310808 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f895c0638a0 con 0x7f89480b6560 2024-09-21T03:00:12.387 INFO:teuthology.orchestra.run.smithi016.stdout:Retention added to path / 2024-09-21T03:00:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 msgr2=0x7f89480b8a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 0x7f89480b8a70 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f8958000c00 tx=0x7f8958031400 comp rx=0 tx=0).stop 2024-09-21T03:00:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 msgr2=0x7f895c19bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f8954009fd0 tx=0x7f895400b260 comp rx=0 tx=0).stop 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 shutdown_connections 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f89480b6560 0x7f89480b8a70 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f895c108c30 0x7f895c19bce0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f895c1082e0 0x7f895c19b7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 --2- 172.21.15.16:0/1983310808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f895c1029b0 0x7f895c19b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 >> 172.21.15.16:0/1983310808 conn(0x7f895c07a740 msgr2=0x7f895c106420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 shutdown_connections 2024-09-21T03:00:12.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.388+0000 7f896393a640 1 -- 172.21.15.16:0/1983310808 wait complete. 2024-09-21T03:00:12.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:00:12.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:12 smithi016 ceph-mon[31253]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:12.423 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:12 smithi016 ceph-mon[31253]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 -- 172.21.15.16:0/3900148798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788108c30 msgr2=0x7fa78810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/3900148798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788108c30 0x7fa78810f460 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fa780009ca0 tx=0x7fa78002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 -- 172.21.15.16:0/3900148798 shutdown_connections 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/3900148798 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa788108c30 0x7fa78810f460 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/3900148798 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 0x7fa7881086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/3900148798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7881029b0 0x7fa788102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 -- 172.21.15.16:0/3900148798 >> 172.21.15.16:0/3900148798 conn(0x7fa78807a740 msgr2=0x7fa78807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 -- 172.21.15.16:0/3900148798 shutdown_connections 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.514+0000 7fa78f2dc640 1 -- 172.21.15.16:0/3900148798 wait complete. 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 Processor -- start 2024-09-21T03:00:12.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 -- start start 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 0x7fa7880ffa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa788108c30 0x7fa7880fffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa7881128f0 con 0x7fa7881029b0 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa788112770 con 0x7fa7881082e0 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78f2dc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa788112a70 con 0x7fa788108c30 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.515+0000 7fa78d078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49776/0 (socket says 172.21.15.16:49776) 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 -- 172.21.15.16:0/599534581 learned_addr learned my addr 172.21.15.16:0/599534581 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78c877640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 0x7fa7880ffa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d879640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa788108c30 0x7fa7880fffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 -- 172.21.15.16:0/599534581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa788108c30 msgr2=0x7fa7880fffd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa788108c30 0x7fa7880fffd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 -- 172.21.15.16:0/599534581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 msgr2=0x7fa7880ffa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 0x7fa7880ffa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 -- 172.21.15.16:0/599534581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7881a8710 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78d078640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa78400e9f0 tx=0x7fa78400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa78400cd80 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7840054d0 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7881a8a00 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa784010690 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.516+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7881a8f00 con 0x7fa7881029b0 2024-09-21T03:00:12.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.517+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa788102dc0 con 0x7fa7881029b0 2024-09-21T03:00:12.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.520+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7fa784004e30 con 0x7fa7881029b0 2024-09-21T03:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.521+0000 7fa77a7fc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 0x7fa7740b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.521+0000 7fa78c877640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 0x7fa7740b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.521+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fa7840e7a80 con 0x7fa7881029b0 2024-09-21T03:00:12.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.521+0000 7fa78c877640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 0x7fa7740b8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0095f0 tx=0x7fa77c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.524+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa7840ae1b0 con 0x7fa7881029b0 2024-09-21T03:00:12.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:12 smithi018 ceph-mon[34587]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:12.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:12 smithi018 ceph-mon[34587]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:12.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:12 smithi123 ceph-mon[34645]: pgmap v316: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:12.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:12 smithi123 ceph-mon[34645]: from='client.15369 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:12.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.739+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7fa788063800 con 0x7fa7740b6590 2024-09-21T03:00:12.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.746+0000 7fa77a7fc640 1 -- 172.21.15.16:0/599534581 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7fa788063800 con 0x7fa7740b6590 2024-09-21T03:00:12.748 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:00:11", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 msgr2=0x7fa7740b8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 0x7fa7740b8aa0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fa77c0095f0 tx=0x7fa77c005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 msgr2=0x7fa7880ff550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7fa78400e9f0 tx=0x7fa78400edf0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 shutdown_connections 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa7740b6590 0x7fa7740b8aa0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa788108c30 0x7fa7880fffd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa7881082e0 0x7fa7880ffa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 --2- 172.21.15.16:0/599534581 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa7881029b0 0x7fa7880ff550 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 >> 172.21.15.16:0/599534581 conn(0x7fa78807a740 msgr2=0x7fa7881019e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 shutdown_connections 2024-09-21T03:00:12.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.749+0000 7fa78f2dc640 1 -- 172.21.15.16:0/599534581 wait complete. 2024-09-21T03:00:12.761 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:00:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.915+0000 7f015c7ad640 1 -- 172.21.15.16:0/1337863366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0154108c30 msgr2=0x7f015410f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.915+0000 7f015c7ad640 1 --2- 172.21.15.16:0/1337863366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0154108c30 0x7f015410f460 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f014c009ca0 tx=0x7f014c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 -- 172.21.15.16:0/1337863366 shutdown_connections 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 --2- 172.21.15.16:0/1337863366 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0154108c30 0x7f015410f460 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 --2- 172.21.15.16:0/1337863366 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541082e0 0x7f01541086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 --2- 172.21.15.16:0/1337863366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01541029b0 0x7f0154102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 -- 172.21.15.16:0/1337863366 >> 172.21.15.16:0/1337863366 conn(0x7f015407a740 msgr2=0x7f015407ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 -- 172.21.15.16:0/1337863366 shutdown_connections 2024-09-21T03:00:12.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 -- 172.21.15.16:0/1337863366 wait complete. 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 Processor -- start 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.916+0000 7f015c7ad640 1 -- start start 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541029b0 0x7f015419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0154108c30 0x7f015419bc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01541128f0 con 0x7f01541082e0 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0154112770 con 0x7f01541029b0 2024-09-21T03:00:12.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0154112a70 con 0x7f0154108c30 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49798/0 (socket says 172.21.15.16:49798) 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 -- 172.21.15.16:0/3163740286 learned_addr learned my addr 172.21.15.16:0/3163740286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015a549640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541029b0 0x7f015419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015ad4a640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0154108c30 0x7f015419bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 -- 172.21.15.16:0/3163740286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0154108c30 msgr2=0x7f015419bc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0154108c30 0x7f015419bc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 -- 172.21.15.16:0/3163740286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541029b0 msgr2=0x7f015419b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541029b0 0x7f015419b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 -- 172.21.15.16:0/3163740286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01541a6410 con 0x7f01541082e0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f0159d48640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f014400c8c0 tx=0x7f014400cd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0144007bb0 con 0x7f01541082e0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0144007d50 con 0x7f01541082e0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01440187c0 con 0x7f01541082e0 2024-09-21T03:00:12.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.917+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01541a6700 con 0x7f01541082e0 2024-09-21T03:00:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.918+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01541a6ca0 con 0x7f01541082e0 2024-09-21T03:00:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.918+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0154102dc0 con 0x7f01541082e0 2024-09-21T03:00:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.920+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 22) ==== 154529+0+0 (secure 0 0 0) 0x7f014400fa20 con 0x7f01541082e0 2024-09-21T03:00:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.921+0000 7f014b7fe640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 0x7f01400b8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:12.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.921+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01440e77a0 con 0x7f01541082e0 2024-09-21T03:00:12.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.921+0000 7f015a549640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 0x7f01400b8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:12.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.922+0000 7f015a549640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 0x7f01400b8af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f015406b760 tx=0x7f0150005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:12.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:12.924+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0144014030 con 0x7f01541082e0 2024-09-21T03:00:13.075 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.137+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f0154063800 con 0x7f01400b65e0 2024-09-21T03:00:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.144+0000 7f014b7fe640 1 -- 172.21.15.16:0/3163740286 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f0154063800 con 0x7f01400b65e0 2024-09-21T03:00:13.146 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.146+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 msgr2=0x7f01400b8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.146+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 0x7f01400b8af0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f015406b760 tx=0x7f0150005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.146+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 msgr2=0x7f015419b720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.146+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f014400c8c0 tx=0x7f014400cd80 comp rx=0 tx=0).stop 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 shutdown_connections 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01400b65e0 0x7f01400b8af0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01541082e0 0x7f015419b720 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0154108c30 0x7f015419bc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 --2- 172.21.15.16:0/3163740286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01541029b0 0x7f015419b1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 >> 172.21.15.16:0/3163740286 conn(0x7f015407a740 msgr2=0x7f01541042b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 shutdown_connections 2024-09-21T03:00:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.147+0000 7f015c7ad640 1 -- 172.21.15.16:0/3163740286 wait complete. 2024-09-21T03:00:13.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-09-21T03:00:13.191 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-09-21T03:00:13.201 INFO:teuthology.run_tasks:Running task workunit... 2024-09-21T03:00:13.245 INFO:tasks.workunit:Pulling workunits from ref 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:00:13.245 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-09-21T03:00:13.245 INFO:tasks.workunit:timeout=3h 2024-09-21T03:00:13.245 INFO:tasks.workunit:cleanup=True 2024-09-21T03:00:13.245 DEBUG:teuthology.orchestra.run.smithi016:> stat -- /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Device: 78h/120d Inode: 1099511628283 Links: 2 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Access: 2024-09-21 02:58:44.239093759 +0000 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Modify: 2024-09-21 02:58:44.239093759 +0000 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout:Change: 2024-09-21 03:00:03.505387532 +0000 2024-09-21T03:00:13.262 INFO:teuthology.orchestra.run.smithi016.stdout: Birth: 2024-09-21 02:58:44.239093759 +0000 2024-09-21T03:00:13.262 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-09-21T03:00:13.262 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-09-21T03:00:13.325 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.326 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:13 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:00:13.335 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 1843221895adccfb3b692060251dd6a0183bc410 2024-09-21T03:00:13.349 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:13.365 INFO:tasks.workunit.client.0.smithi016.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-09-21T03:00:13.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:00:13.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:00:13.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:13 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:00:13.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:00:13.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:00:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: from='client.15381 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: from='client.15405 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:13 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.886+0000 7fe758a18640 1 -- 172.21.15.16:0/3856928159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7541094a0 msgr2=0x7fe7541098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.886+0000 7fe758a18640 1 --2- 172.21.15.16:0/3856928159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7541094a0 0x7fe7541098b0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7fe748009ca0 tx=0x7fe74802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 -- 172.21.15.16:0/3856928159 shutdown_connections 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 --2- 172.21.15.16:0/3856928159 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe7541094a0 0x7fe7541098b0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 --2- 172.21.15.16:0/3856928159 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe754109df0 0x7fe754110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 --2- 172.21.15.16:0/3856928159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe754103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 -- 172.21.15.16:0/3856928159 >> 172.21.15.16:0/3856928159 conn(0x7fe75407b850 msgr2=0x7fe75407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 -- 172.21.15.16:0/3856928159 shutdown_connections 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.887+0000 7fe758a18640 1 -- 172.21.15.16:0/3856928159 wait complete. 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 Processor -- start 2024-09-21T03:00:13.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 -- start start 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe75419a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7541094a0 0x7fe75419aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe754113ab0 con 0x7fe754109df0 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe754113930 con 0x7fe7541094a0 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe758a18640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe754113c30 con 0x7fe754103b60 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe752d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49828/0 (socket says 172.21.15.16:49828) 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.888+0000 7fe752d9d640 1 -- 172.21.15.16:0/4289545715 learned_addr learned my addr 172.21.15.16:0/4289545715 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe751d9b640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7541094a0 0x7fe75419aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe75259c640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe75419a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 -- 172.21.15.16:0/4289545715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 msgr2=0x7fe75419a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe75419a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 -- 172.21.15.16:0/4289545715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7541094a0 msgr2=0x7fe75419aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7541094a0 0x7fe75419aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 -- 172.21.15.16:0/4289545715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe7541a0410 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe75259c640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe75419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe752d9d640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00d750 tx=0x7fe74c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe7437fe640 1 -- 172.21.15.16:0/4289545715 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c017070 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe7541a0700 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe7437fe640 1 -- 172.21.15.16:0/4289545715 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe74c0054d0 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe7437fe640 1 -- 172.21.15.16:0/4289545715 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74c004c60 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.889+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7541a6e60 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.892+0000 7fe75ef8c740 1 Processor -- start 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.892+0000 7fe7437fe640 1 -- 172.21.15.16:0/4289545715 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fe74c010460 con 0x7fe754109df0 2024-09-21T03:00:13.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.892+0000 7fe75ef8c740 1 -- start start 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9008270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa9008cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfa8f7ee50 con 0x55cfa8c381b0 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfa8f6e280 con 0x55cfa8c61070 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cfa8f72090 con 0x55cfa8c40390 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49836/0 (socket says 172.21.15.16:49836) 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 -- 172.21.15.16:0/3386420333 learned_addr learned my addr 172.21.15.16:0/3386420333 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:13.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9008270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa9008cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe7437fe640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 0x7fe73c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 -- 172.21.15.16:0/3386420333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 msgr2=0x55cfa9008cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa9008cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 -- 172.21.15.16:0/3386420333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 msgr2=0x55cfa9008270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9008270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.893+0000 7fe75259c640 1 -- 172.21.15.16:0/3386420333 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa8ccc650 con 0x55cfa8c381b0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe7437fe640 1 -- 172.21.15.16:0/4289545715 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe74c0e70b0 con 0x7fe754109df0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe75259c640 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe74400bb70 tx=0x7fe74400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0f8c0b4528ceb9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe752d9d640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 0x7fe73c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe740ff9640 1 -- 172.21.15.16:0/3386420333 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744017620 con 0x55cfa8c381b0 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe740ff9640 1 -- 172.21.15.16:0/3386420333 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe744017c70 con 0x55cfa8c381b0 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.894+0000 7fe740ff9640 1 -- 172.21.15.16:0/3386420333 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe74400fa60 con 0x55cfa8c381b0 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3386420333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 msgr2=0x55cfa90087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fe74400bb70 tx=0x7fe74400d710 comp rx=0 tx=0).stop 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe752d9d640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 0x7fe73c0b8b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0ad010 tx=0x7fe74c005bb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3386420333 shutdown_connections 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa90087b0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa9008cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3386420333 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9008270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3386420333 >> 172.21.15.16:0/3386420333 conn(0x55cfa8c55460 msgr2=0x55cfa9006160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3386420333 shutdown_connections 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.895+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3386420333 wait complete. 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 -- start start 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 Processor -- start 2024-09-21T03:00:13.902 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c381b0 0x55cfa9017cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9018730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55cfa8dcf700 con 0x55cfa8c40390 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55cfa8f09110 con 0x55cfa8c61070 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55cfa901adb0 con 0x55cfa8c381b0 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75ef8c740 10 client.0 _ll_register_callbacks cb 0x55cfa8f22990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49850/0 (socket says 172.21.15.16:49850) 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe75259c640 1 -- 172.21.15.16:0/2680539548 learned_addr learned my addr 172.21.15.16:0/2680539548 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe751d9b640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c381b0 0x55cfa9017cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.896+0000 7fe751d9b640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9018730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 -- 172.21.15.16:0/2680539548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c381b0 msgr2=0x55cfa9017cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c381b0 0x55cfa9017cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.903 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 -- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 msgr2=0x55cfa9018730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9018730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 -- 172.21.15.16:0/2680539548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa8f36c40 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe751d9b640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9018730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75259c640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b310 tx=0x7fe744005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5a3d684982a9e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744026070 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe744017c70 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe744017830 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe744032050 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75ef8c740 10 client.15423 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.897+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cfa9026a40 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe744017c70 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fe744022ab0 con 0x55cfa8c40390 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 10 client.15423 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:13.904 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 10 client.15423 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 20 client.15423 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8c1f9f0 0x55cfa9029f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 4 client.15423 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cfa8f80da0 con 0x55cfa8c1f9f0 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.898+0000 7fe751d9b640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8c1f9f0 0x55cfa9029f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.899+0000 7fe751d9b640 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8c1f9f0 0x55cfa9029f50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=66a2f7090f86b520 server_cookie=d0575a226c631bab in_seq=0 out_seq=0 2024-09-21T03:00:13.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.899+0000 7fe739ffb640 10 client.15423 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.899+0000 7fe739ffb640 1 -- 172.21.15.16:0/2680539548 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55cfa8f80da0 con 0x55cfa8c1f9f0 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.899+0000 7fe739ffb640 10 client.15423 handle_command_reply: tid=0 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.899+0000 7fe75ef8c740 1 client.15423 shutdown 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 msgr2=0x55cfa90181f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fe74400b310 tx=0x7fe744005bf0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 shutdown_connections 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8c1f9f0 0x55cfa9029f50 crc :-1 s=READY pgs=25 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa90181f0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.906 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9018730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/2680539548 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c381b0 0x55cfa9017cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe739ffb640 0 client.15423 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 >> 172.21.15.16:0/2680539548 conn(0x55cfa8c55460 msgr2=0x55cfa9004700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 shutdown_connections 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 1 -- 172.21.15.16:0/2680539548 wait complete. 2024-09-21T03:00:13.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.900+0000 7fe75ef8c740 20 client.15423 trim_cache size 0 max 16384 2024-09-21T03:00:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 Processor -- start 2024-09-21T03:00:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 -- start start 2024-09-21T03:00:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa90198e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9014ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541ab550 con 0x55cfa8c381b0 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541addd0 con 0x55cfa8c61070 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541abbc0 con 0x55cfa8c40390 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe751d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe751d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49856/0 (socket says 172.21.15.16:49856) 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.918+0000 7fe751d9b640 1 -- 172.21.15.16:0/1240693362 learned_addr learned my addr 172.21.15.16:0/1240693362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 -- 172.21.15.16:0/1240693362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 msgr2=0x55cfa90198e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa90198e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 -- 172.21.15.16:0/1240693362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 msgr2=0x55cfa9014ea0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9014ea0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 -- 172.21.15.16:0/1240693362 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cfa901a3f0 con 0x55cfa8c381b0 2024-09-21T03:00:13.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe751d9b640 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe748000f80 tx=0x7fe7480095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d86ef1115571120e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe73bfff640 1 -- 172.21.15.16:0/1240693362 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748031af0 con 0x55cfa8c381b0 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe73bfff640 1 -- 172.21.15.16:0/1240693362 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe748036040 con 0x55cfa8c381b0 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.919+0000 7fe73bfff640 1 -- 172.21.15.16:0/1240693362 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe748003cf0 con 0x55cfa8c381b0 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 -- 172.21.15.16:0/1240693362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 msgr2=0x55cfa9009c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7fe748000f80 tx=0x7fe7480095c0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 -- 172.21.15.16:0/1240693362 shutdown_connections 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c61070 0x55cfa9014ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c40390 0x55cfa90198e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/1240693362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c381b0 0x55cfa9009c60 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 -- 172.21.15.16:0/1240693362 >> 172.21.15.16:0/1240693362 conn(0x55cfa8c55460 msgr2=0x55cfa90191d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 -- 172.21.15.16:0/1240693362 shutdown_connections 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.920+0000 7fe75ef8c740 1 -- 172.21.15.16:0/1240693362 wait complete. 2024-09-21T03:00:13.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 -- start start 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 Processor -- start 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c381b0 0x55cfa901c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 0x55cfa9093760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541ac290 con 0x55cfa8c40390 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541ad700 con 0x55cfa8c381b0 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe7541ac960 con 0x55cfa8c61070 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75ef8c740 10 client.0 _ll_register_callbacks cb 0x7fe74400b9c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75259c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49866/0 (socket says 172.21.15.16:49866) 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75259c640 1 -- 172.21.15.16:0/3830345634 learned_addr learned my addr 172.21.15.16:0/3830345634 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 0x55cfa9093760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c381b0 0x55cfa901c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.921+0000 7fe75259c640 1 -- 172.21.15.16:0/3830345634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 msgr2=0x55cfa9093760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75259c640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 0x55cfa9093760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75259c640 1 -- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c381b0 msgr2=0x55cfa901c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75259c640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c381b0 0x55cfa901c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75259c640 1 -- 172.21.15.16:0/3830345634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cfa901d170 con 0x55cfa8c40390 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 0x55cfa9093760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75259c640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe744005fe0 tx=0x7fe744005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d2f8f3506084661 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe739ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe744026070 con 0x55cfa8c40390 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe748036040 con 0x55cfa8c40390 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe744020980 con 0x55cfa8c40390 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe74402f020 con 0x55cfa8c40390 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75ef8c740 10 client.15429 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:13.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55cfa9094d30 con 0x55cfa8c40390 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.922+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe748036040 con 0x55cfa8c40390 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fe744020bf0 con 0x55cfa8c40390 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 10 client.15429 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 10 client.15429 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 20 client.15429 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8bf4da0 0x55cfa90973f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 4 client.15429 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cfa8f7c850 con 0x55cfa8bf4da0 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8bf4da0 0x55cfa90973f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe751d9b640 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8bf4da0 0x55cfa90973f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eaa4c1be22c7fdbe server_cookie=aa46ef4e84f9140b in_seq=0 out_seq=0 2024-09-21T03:00:13.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.923+0000 7fe739ffb640 10 client.15429 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe739ffb640 1 -- 172.21.15.16:0/3830345634 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+374 (crc 0 0 0) 0x55cfa8f7c850 con 0x55cfa8bf4da0 2024-09-21T03:00:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe739ffb640 10 client.15429 handle_command_reply: tid=0 2024-09-21T03:00:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 client.15429 shutdown 2024-09-21T03:00:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 msgr2=0x55cfa901c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fe744005fe0 tx=0x7fe744005e30 comp rx=0 tx=0).stop 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 shutdown_connections 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55cfa8bf4da0 0x55cfa90973f0 crc :-1 s=READY pgs=26 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55cfa8c40390 0x55cfa901c8a0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55cfa8c61070 0x55cfa9093760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 --2- 172.21.15.16:0/3830345634 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55cfa8c381b0 0x55cfa901c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe739ffb640 0 client.15429 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 >> 172.21.15.16:0/3830345634 conn(0x55cfa8c55460 msgr2=0x55cfa9009a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 shutdown_connections 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 1 -- 172.21.15.16:0/3830345634 wait complete. 2024-09-21T03:00:13.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.924+0000 7fe75ef8c740 20 client.15429 trim_cache size 0 max 16384 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:13.931 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 msgr2=0x7fe73c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 0x7fe73c0b8b00 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe74c0ad010 tx=0x7fe74c005bb0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 msgr2=0x7fe75419fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fe74c00d750 tx=0x7fe74c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 shutdown_connections 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe73c0b65f0 0x7fe73c0b8b00 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe754109df0 0x7fe75419fce0 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe7541094a0 0x7fe75419aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 --2- 172.21.15.16:0/4289545715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe754103b60 0x7fe75419a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 >> 172.21.15.16:0/4289545715 conn(0x7fe75407b850 msgr2=0x7fe754105510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 shutdown_connections 2024-09-21T03:00:13.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:13.925+0000 7fe758a18640 1 -- 172.21.15.16:0/4289545715 wait complete. 2024-09-21T03:00:14.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:14 smithi016 ceph-mon[31253]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:14.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:14 smithi016 ceph-mon[31253]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:14.657 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:14 smithi016 ceph-mon[31253]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:00:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:14 smithi018 ceph-mon[34587]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:14 smithi018 ceph-mon[34587]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:14 smithi018 ceph-mon[34587]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:00:14.690 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:00:14.691 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:14 smithi123 ceph-mon[34645]: from='client.15411 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:00:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:14 smithi123 ceph-mon[34645]: pgmap v317: 97 pgs: 97 active+clean; 649 KiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:14 smithi123 ceph-mon[34645]: mgrmap e23: x(active, since 8m), standbys: z, y 2024-09-21T03:00:14.996 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 -- 172.21.15.16:0/556925341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc1083a0 msgr2=0x7ff2dc1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/556925341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1087b0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7ff2d0009900 tx=0x7ff2d002a410 comp rx=0 tx=0).stop 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 -- 172.21.15.16:0/556925341 shutdown_connections 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/556925341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc1087b0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/556925341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/556925341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc102a60 0x7ff2dc102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 -- 172.21.15.16:0/556925341 >> 172.21.15.16:0/556925341 conn(0x7ff2dc079730 msgr2=0x7ff2dc079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 -- 172.21.15.16:0/556925341 shutdown_connections 2024-09-21T03:00:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.509+0000 7ff2e13c8640 1 -- 172.21.15.16:0/556925341 wait complete. 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 Processor -- start 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 -- start start 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc102a60 0x7ff2dc19b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc19b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112960 con 0x7ff2dc108cf0 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc1127e0 con 0x7ff2dc102a60 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2e13c8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff2dc112ae0 con 0x7ff2dc1083a0 2024-09-21T03:00:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2db7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39522/0 (socket says 172.21.15.16:39522) 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2da7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc19b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2db7fe640 1 -- 172.21.15.16:0/362560796 learned_addr learned my addr 172.21.15.16:0/362560796 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.510+0000 7ff2daffd640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc102a60 0x7ff2dc19b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 -- 172.21.15.16:0/362560796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1083a0 msgr2=0x7ff2dc19b7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc19b7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 -- 172.21.15.16:0/362560796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc102a60 msgr2=0x7ff2dc19b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc102a60 0x7ff2dc19b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 -- 172.21.15.16:0/362560796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff2dc1a6520 con 0x7ff2dc108cf0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2db7fe640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400eb10 tx=0x7ff2d400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4004b20 con 0x7ff2dc108cf0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff2d4004cc0 con 0x7ff2dc108cf0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff2dc1a6810 con 0x7ff2dc108cf0 2024-09-21T03:00:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff2d4010670 con 0x7ff2dc108cf0 2024-09-21T03:00:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.511+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff2dc1a6d70 con 0x7ff2dc108cf0 2024-09-21T03:00:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.512+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff2dc102e70 con 0x7ff2dc108cf0 2024-09-21T03:00:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.514+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7ff2d4004e70 con 0x7ff2dc108cf0 2024-09-21T03:00:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.515+0000 7ff2c3fff640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 0x7ff2c40b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.515+0000 7ff2daffd640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 0x7ff2c40b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.515+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7ff2d40e7d10 con 0x7ff2dc108cf0 2024-09-21T03:00:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.516+0000 7ff2daffd640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 0x7ff2c40b8bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc06b9c0 tx=0x7ff2cc00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.518+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff2d4056710 con 0x7ff2dc108cf0 2024-09-21T03:00:15.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.744+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff2dc06b270 con 0x7ff2dc108cf0 2024-09-21T03:00:15.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.745+0000 7ff2c3fff640 1 -- 172.21.15.16:0/362560796 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7ff2d40ae2e0 con 0x7ff2dc108cf0 2024-09-21T03:00:15.746 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:15.747 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:00:15.747 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:00:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 msgr2=0x7ff2c40b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 0x7ff2c40b8bb0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff2dc06b9c0 tx=0x7ff2cc00b390 comp rx=0 tx=0).stop 2024-09-21T03:00:15.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 msgr2=0x7ff2dc19bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff2d400eb10 tx=0x7ff2d400c2d0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 shutdown_connections 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff2c40b66a0 0x7ff2c40b8bb0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff2dc108cf0 0x7ff2dc19bd10 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2dc1083a0 0x7ff2dc19b7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 --2- 172.21.15.16:0/362560796 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff2dc102a60 0x7ff2dc19b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 >> 172.21.15.16:0/362560796 conn(0x7ff2dc079730 msgr2=0x7ff2dc104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 shutdown_connections 2024-09-21T03:00:15.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:15.747+0000 7ff2e13c8640 1 -- 172.21.15.16:0/362560796 wait complete. 2024-09-21T03:00:16.498 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:16 smithi016 ceph-mon[31253]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 50 op/s 2024-09-21T03:00:16.499 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:16 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/362560796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:16.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:00:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:16 smithi018 ceph-mon[34587]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 50 op/s 2024-09-21T03:00:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:16 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/362560796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:16.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:16 smithi123 ceph-mon[34645]: pgmap v318: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 50 op/s 2024-09-21T03:00:16.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:16 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/362560796' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:16.795 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:17.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 -- 172.21.15.16:0/2003885825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb268103b60 msgr2=0x7fb268103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/2003885825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb268103b60 0x7fb268103f70 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fb258009900 tx=0x7fb25802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 -- 172.21.15.16:0/2003885825 shutdown_connections 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/2003885825 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268109df0 0x7fb268110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/2003885825 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2681094a0 0x7fb2681098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/2003885825 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb268103b60 0x7fb268103f70 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 -- 172.21.15.16:0/2003885825 >> 172.21.15.16:0/2003885825 conn(0x7fb26807b850 msgr2=0x7fb26807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 -- 172.21.15.16:0/2003885825 shutdown_connections 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.278+0000 7fb26fcd5640 1 -- 172.21.15.16:0/2003885825 wait complete. 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.279+0000 7fb26fcd5640 1 Processor -- start 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.279+0000 7fb26fcd5640 1 -- start start 2024-09-21T03:00:17.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 0x7fb268077120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb268109df0 0x7fb268075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb268113ab0 con 0x7fb2681094a0 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb268113930 con 0x7fb268103b60 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26fcd5640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb268113c30 con 0x7fb268109df0 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26da71640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 0x7fb268077120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39532/0 (socket says 172.21.15.16:39532) 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26da71640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 0x7fb268077120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:42178/0 (socket says 172.21.15.16:42178) 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 -- 172.21.15.16:0/4096438967 learned_addr learned my addr 172.21.15.16:0/4096438967 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26e272640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb268109df0 0x7fb268075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 -- 172.21.15.16:0/4096438967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb268109df0 msgr2=0x7fb268075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb268109df0 0x7fb268075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 -- 172.21.15.16:0/4096438967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 msgr2=0x7fb268077120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 0x7fb268077120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 -- 172.21.15.16:0/4096438967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb268076470 con 0x7fb2681094a0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.280+0000 7fb26d270640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d750 tx=0x7fb25c00dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.281+0000 7fb266ffd640 1 -- 172.21.15.16:0/4096438967 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c017070 con 0x7fb2681094a0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.281+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2681acea0 con 0x7fb2681094a0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.281+0000 7fb266ffd640 1 -- 172.21.15.16:0/4096438967 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb25c0054d0 con 0x7fb2681094a0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.281+0000 7fb266ffd640 1 -- 172.21.15.16:0/4096438967 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c004c30 con 0x7fb2681094a0 2024-09-21T03:00:17.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.281+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2681ad2f0 con 0x7fb2681094a0 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.283+0000 7fb276249740 1 Processor -- start 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.283+0000 7fb266ffd640 1 -- 172.21.15.16:0/4096438967 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fb25c004dd0 con 0x7fb2681094a0 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 -- start start 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134c7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c61343dea0 con 0x55c613120070 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c61342d2d0 con 0x55c6130ff390 2024-09-21T03:00:17.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb276249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c6134310e0 con 0x55c6130f71b0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39544/0 (socket says 172.21.15.16:39544) 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb26e272640 1 -- 172.21.15.16:0/798600390 learned_addr learned my addr 172.21.15.16:0/798600390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb26da71640 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134c7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.284+0000 7fb26d270640 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 -- 172.21.15.16:0/798600390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 msgr2=0x55c6134c7800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134c7800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 -- 172.21.15.16:0/798600390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 msgr2=0x55c6134c7d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134c7d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 -- 172.21.15.16:0/798600390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c61318b650 con 0x55c613120070 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb266ffd640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 0x7fb2500b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26e272640 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb260009fc0 tx=0x7fb260010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe739578b321199 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb26da71640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 0x7fb2500b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb266ffd640 1 -- 172.21.15.16:0/4096438967 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb25c0e6f60 con 0x7fb2681094a0 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb24ffff640 1 -- 172.21.15.16:0/798600390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260016070 con 0x55c613120070 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb24ffff640 1 -- 172.21.15.16:0/798600390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb260004d20 con 0x55c613120070 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.285+0000 7fb24ffff640 1 -- 172.21.15.16:0/798600390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb2600053e0 con 0x55c613120070 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 -- 172.21.15.16:0/798600390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 msgr2=0x55c6134c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb26da71640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 0x7fb2500b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb258000c00 tx=0x7fb258031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7fb260009fc0 tx=0x7fb260010800 comp rx=0 tx=0).stop 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 -- 172.21.15.16:0/798600390 shutdown_connections 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134c7d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134c7800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 --2- 172.21.15.16:0/798600390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134c72c0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 -- 172.21.15.16:0/798600390 >> 172.21.15.16:0/798600390 conn(0x55c613114460 msgr2=0x55c6134c51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 -- 172.21.15.16:0/798600390 shutdown_connections 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.286+0000 7fb276249740 1 -- 172.21.15.16:0/798600390 wait complete. 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 -- start start 2024-09-21T03:00:17.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 Processor -- start 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134d29a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134d2ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55c61328e700 con 0x55c613120070 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55c6133c8160 con 0x55c6130ff390 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55c6134d5aa0 con 0x55c6130f71b0 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb276249740 10 client.0 _ll_register_callbacks cb 0x55c6133e19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39554/0 (socket says 172.21.15.16:39554) 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb26e272640 1 -- 172.21.15.16:0/934103787 learned_addr learned my addr 172.21.15.16:0/934103787 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb26d270640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134d2ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.287+0000 7fb26da71640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134d29a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 -- 172.21.15.16:0/934103787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 msgr2=0x55c6134d29a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134d29a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 -- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 msgr2=0x55c6134d2ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134d2ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 -- 172.21.15.16:0/934103787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6133f5c90 con 0x55c613120070 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb26e272640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb260000f80 tx=0x7fb260009ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b294ba65bcc52c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb24d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260029060 con 0x55c613120070 2024-09-21T03:00:17.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb260004d20 con 0x55c613120070 2024-09-21T03:00:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.288+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb260012510 con 0x55c613120070 2024-09-21T03:00:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.289+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb260016070 con 0x55c613120070 2024-09-21T03:00:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.289+0000 7fb276249740 10 client.15447 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:17.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.289+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c6134e17e0 con 0x55c613120070 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb260004d20 con 0x55c613120070 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb26000f040 con 0x55c613120070 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 10 client.15447 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 10 client.15447 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 20 client.15447 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c6130de9f0 0x55c6134e4c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 4 client.15447 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c61343fdf0 con 0x55c6130de9f0 2024-09-21T03:00:17.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.292+0000 7fb26da71640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c6130de9f0 0x55c6134e4c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.293+0000 7fb24d7fa640 10 client.15447 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.293+0000 7fb26da71640 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c6130de9f0 0x55c6134e4c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f67a3f45cdd8ed server_cookie=c38f55c16053e5df in_seq=0 out_seq=0 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb24d7fa640 1 -- 172.21.15.16:0/934103787 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55c61343fdf0 con 0x55c6130de9f0 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb24d7fa640 10 client.15447 handle_command_reply: tid=0 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 client.15447 shutdown 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 msgr2=0x55c6134ded90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb260000f80 tx=0x7fb260009ce0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 shutdown_connections 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c6130de9f0 0x55c6134e4c50 crc :-1 s=READY pgs=27 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c613120070 0x55c6134ded90 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134d2ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb24d7fa640 0 client.15447 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 --2- 172.21.15.16:0/934103787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130f71b0 0x55c6134d29a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 >> 172.21.15.16:0/934103787 conn(0x55c613114460 msgr2=0x55c6134ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 shutdown_connections 2024-09-21T03:00:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 1 -- 172.21.15.16:0/934103787 wait complete. 2024-09-21T03:00:17.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.294+0000 7fb276249740 20 client.15447 trim_cache size 0 max 16384 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 Processor -- start 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 -- start start 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134c8a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130f71b0 0x55c6134ded00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b3c90 con 0x55c6130ff390 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b6510 con 0x55c6130f71b0 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.308+0000 7fb276249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b4300 con 0x55c6130de9f0 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39562/0 (socket says 172.21.15.16:39562) 2024-09-21T03:00:17.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 -- 172.21.15.16:0/3019584839 learned_addr learned my addr 172.21.15.16:0/3019584839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26d270640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130f71b0 0x55c6134ded00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26da71640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134c8a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 -- 172.21.15.16:0/3019584839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 msgr2=0x55c6134c8a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134c8a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 -- 172.21.15.16:0/3019584839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130f71b0 msgr2=0x55c6134ded00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130f71b0 0x55c6134ded00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 -- 172.21.15.16:0/3019584839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6134c95b0 con 0x55c6130ff390 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26da71640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb26e272640 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb260009770 tx=0x7fb260005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9062fe0c64d9a7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb24f7fe640 1 -- 172.21.15.16:0/3019584839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260029040 con 0x55c6130ff390 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb24f7fe640 1 -- 172.21.15.16:0/3019584839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb260005910 con 0x55c6130ff390 2024-09-21T03:00:17.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.309+0000 7fb24f7fe640 1 -- 172.21.15.16:0/3019584839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb260003fc0 con 0x55c6130ff390 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 -- 172.21.15.16:0/3019584839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 msgr2=0x55c6134cfb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb260009770 tx=0x7fb260005bd0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 -- 172.21.15.16:0/3019584839 shutdown_connections 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130ff390 0x55c6134cfb90 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130f71b0 0x55c6134ded00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 --2- 172.21.15.16:0/3019584839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134c8a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 -- 172.21.15.16:0/3019584839 >> 172.21.15.16:0/3019584839 conn(0x55c613114460 msgr2=0x55c6134c82f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 -- 172.21.15.16:0/3019584839 shutdown_connections 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.310+0000 7fb276249740 1 -- 172.21.15.16:0/3019584839 wait complete. 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 -- start start 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 Processor -- start 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134e8560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134e6060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b49d0 con 0x55c6130f71b0 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b5e40 con 0x55c6130ff390 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb2681b50a0 con 0x55c6130de9f0 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb276249740 10 client.0 _ll_register_callbacks cb 0x7fb258009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb26d270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb26d270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39574/0 (socket says 172.21.15.16:39574) 2024-09-21T03:00:17.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.311+0000 7fb26d270640 1 -- 172.21.15.16:0/1790470305 learned_addr learned my addr 172.21.15.16:0/1790470305 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26e272640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134e6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26da71640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134e8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 -- 172.21.15.16:0/1790470305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 msgr2=0x55c6134e8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134e8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 -- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 msgr2=0x55c6134e6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134e6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 -- 172.21.15.16:0/1790470305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6134e67f0 con 0x55c6130f71b0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26da71640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb26d270640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0aced0 tx=0x7fb25c0a8ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c870481d655f4f1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb24d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb25c020040 con 0x55c6130f71b0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb25c0940b0 con 0x55c6130f71b0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb260005910 con 0x55c6130f71b0 2024-09-21T03:00:17.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb25c0aca50 con 0x55c6130f71b0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb276249740 10 client.15453 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.312+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55c61354f380 con 0x55c6130f71b0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb260005910 con 0x55c6130f71b0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb25c0a8890 con 0x55c6130f71b0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 10 client.15453 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 10 client.15453 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 20 client.15453 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c613120070 0x55c613551b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 4 client.15453 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c61343b8a0 con 0x55c613120070 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.313+0000 7fb26da71640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c613120070 0x55c613551b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb26da71640 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c613120070 0x55c613551b60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ad173b82ec39c77 server_cookie=366edc2af6b66090 in_seq=0 out_seq=0 2024-09-21T03:00:17.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb24d7fa640 10 client.15453 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb24d7fa640 1 -- 172.21.15.16:0/1790470305 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c61343b8a0 con 0x55c613120070 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb24d7fa640 10 client.15453 handle_command_reply: tid=0 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 client.15453 shutdown 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 msgr2=0x55c6134e5a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7fb25c0aced0 tx=0x7fb25c0a8ea0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 shutdown_connections 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55c613120070 0x55c613551b60 crc :-1 s=READY pgs=28 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55c6130f71b0 0x55c6134e5a30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55c6130ff390 0x55c6134e6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 --2- 172.21.15.16:0/1790470305 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c6130de9f0 0x55c6134e8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb24d7fa640 0 client.15453 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 >> 172.21.15.16:0/1790470305 conn(0x55c613114460 msgr2=0x55c6134d2ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 shutdown_connections 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 1 -- 172.21.15.16:0/1790470305 wait complete. 2024-09-21T03:00:17.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.314+0000 7fb276249740 20 client.15453 trim_cache size 0 max 16384 2024-09-21T03:00:17.321 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:00:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 msgr2=0x7fb2500b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 0x7fb2500b8b00 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fb258000c00 tx=0x7fb258031400 comp rx=0 tx=0).stop 2024-09-21T03:00:17.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 msgr2=0x7fb268075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7fb25c00d750 tx=0x7fb25c00dc10 comp rx=0 tx=0).stop 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 shutdown_connections 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb2500b65f0 0x7fb2500b8b00 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb2681094a0 0x7fb268075710 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb268109df0 0x7fb268075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 --2- 172.21.15.16:0/4096438967 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb268103b60 0x7fb268077120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 >> 172.21.15.16:0/4096438967 conn(0x7fb26807b850 msgr2=0x7fb268107590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 shutdown_connections 2024-09-21T03:00:17.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:17.315+0000 7fb26fcd5640 1 -- 172.21.15.16:0/4096438967 wait complete. 2024-09-21T03:00:18.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:18 smithi018 ceph-mon[34587]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:18.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:18 smithi123 ceph-mon[34645]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:18.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:18 smithi016 ceph-mon[31253]: pgmap v319: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:19.048 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:00:19.048 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:00:19.319 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 -- 172.21.15.16:0/597243194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac105290 msgr2=0x7fb4ac10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/597243194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac105290 0x7fb4ac10d8d0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0009900 tx=0x7fb4a002a410 comp rx=0 tx=0).stop 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 -- 172.21.15.16:0/597243194 shutdown_connections 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/597243194 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac105290 0x7fb4ac10d8d0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/597243194 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/597243194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac104860 0x7fb4ac104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 -- 172.21.15.16:0/597243194 >> 172.21.15.16:0/597243194 conn(0x7fb4ac079740 msgr2=0x7fb4ac079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 -- 172.21.15.16:0/597243194 shutdown_connections 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.849+0000 7fb4b0e52640 1 -- 172.21.15.16:0/597243194 wait complete. 2024-09-21T03:00:19.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 Processor -- start 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 -- start start 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 0x7fb4ac19aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac19fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac113a90 con 0x7fb4ac104860 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac113910 con 0x7fb4ac10de10 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4b0e52640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac113c10 con 0x7fb4ac105290 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39588/0 (socket says 172.21.15.16:39588) 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 -- 172.21.15.16:0/154315861 learned_addr learned my addr 172.21.15.16:0/154315861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 0x7fb4ac19aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 -- 172.21.15.16:0/154315861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 msgr2=0x7fb4ac19aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 0x7fb4ac19aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac19fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 -- 172.21.15.16:0/154315861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 msgr2=0x7fb4ac19fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac19fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.850+0000 7fb4aa59c640 1 -- 172.21.15.16:0/154315861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4ac1a0450 con 0x7fb4ac104860 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 0x7fb4ac19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00d770 tx=0x7fb49c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb49b7fe640 1 -- 172.21.15.16:0/154315861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c017070 con 0x7fb4ac104860 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4ac1a0740 con 0x7fb4ac104860 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb49b7fe640 1 -- 172.21.15.16:0/154315861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb49c009c30 con 0x7fb4ac104860 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb49b7fe640 1 -- 172.21.15.16:0/154315861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb49c003cc0 con 0x7fb4ac104860 2024-09-21T03:00:19.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.851+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4ac1a6ea0 con 0x7fb4ac104860 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 Processor -- start 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb49b7fe640 1 -- 172.21.15.16:0/154315861 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fb49c003e60 con 0x7fb4ac104860 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 -- start start 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4956b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4956b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac494e1e60 con 0x55ac491a3670 2024-09-21T03:00:19.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ac494d1290 con 0x55ac491c4350 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac494d50a0 con 0x55ac4919b490 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39590/0 (socket says 172.21.15.16:39590) 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/3820288795 learned_addr learned my addr 172.21.15.16:0/3820288795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.854+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4956b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4956b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/3820288795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 msgr2=0x55ac4956b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4956b970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/3820288795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 msgr2=0x55ac4956b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb49b7fe640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4956b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/3820288795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4922f930 con 0x55ac491a3670 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0032ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ee134c80b1cf152 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb49b7fe640 1 -- 172.21.15.16:0/154315861 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb49c0e7020 con 0x7fb4ac104860 2024-09-21T03:00:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb498ff9640 1 -- 172.21.15.16:0/3820288795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a003f070 con 0x55ac491a3670 2024-09-21T03:00:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb498ff9640 1 -- 172.21.15.16:0/3820288795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a00316a0 con 0x55ac491a3670 2024-09-21T03:00:19.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.855+0000 7fb498ff9640 1 -- 172.21.15.16:0/3820288795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a0035070 con 0x55ac491a3670 2024-09-21T03:00:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 -- 172.21.15.16:0/3820288795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 msgr2=0x55ac4956beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 0x7fb4940b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a4009680 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7fb4a0000c00 tx=0x7fb4a0032ce0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 -- 172.21.15.16:0/3820288795 shutdown_connections 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4956beb0 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4956b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/3820288795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4956b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.856+0000 7fb4b73c6740 1 -- 172.21.15.16:0/3820288795 >> 172.21.15.16:0/3820288795 conn(0x55ac491b8740 msgr2=0x55ac49569320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- 172.21.15.16:0/3820288795 shutdown_connections 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- 172.21.15.16:0/3820288795 wait complete. 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- start start 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 Processor -- start 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 0x55ac49577030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac491c4350 0x55ac49582ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ac49332890 con 0x55ac491a3670 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ac4946bd40 con 0x55ac4919b490 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.857+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ac49579bb0 con 0x55ac491c4350 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4b73c6740 10 client.0 _ll_register_callbacks cb 0x55ac494cfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39592/0 (socket says 172.21.15.16:39592) 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/1196651498 learned_addr learned my addr 172.21.15.16:0/1196651498 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 0x55ac49577030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac491c4350 0x55ac49582ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/1196651498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac491c4350 msgr2=0x55ac49582ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac491c4350 0x55ac49582ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 msgr2=0x55ac49577030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 0x55ac49577030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/1196651498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac495834e0 con 0x55ac491a3670 2024-09-21T03:00:19.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 0x55ac49577030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002a450 tx=0x7fb4a0005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388beef373ba6a30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a003f070 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb4a0004230 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.858+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a00316a0 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb4a0053050 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 10 client.15465 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ac49585e10 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a00316a0 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb4a003b460 con 0x55ac491a3670 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 10 client.15465 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 10 client.15465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:19.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 20 client.15465 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49182cd0 0x55ac49589170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 4 client.15465 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac494e3db0 con 0x55ac49182cd0 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.859+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49182cd0 0x55ac49589170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.860+0000 7fb491ffb640 10 client.15465 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.860+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49182cd0 0x55ac49589170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c6efe35c42b32924 server_cookie=ec7b7266073d0831 in_seq=0 out_seq=0 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.860+0000 7fb491ffb640 1 -- 172.21.15.16:0/1196651498 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ac494e3db0 con 0x55ac49182cd0 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.860+0000 7fb491ffb640 10 client.15465 handle_command_reply: tid=0 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 client.15465 shutdown 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 msgr2=0x55ac495775b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fb4a002a450 tx=0x7fb4a0005f30 comp rx=0 tx=0).stop 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 shutdown_connections 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49182cd0 0x55ac49589170 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac495775b0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac491c4350 0x55ac49582ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/1196651498 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac4919b490 0x55ac49577030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 >> 172.21.15.16:0/1196651498 conn(0x55ac491b8740 msgr2=0x55ac4956e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb491ffb640 0 client.15465 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 shutdown_connections 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 1 -- 172.21.15.16:0/1196651498 wait complete. 2024-09-21T03:00:19.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.861+0000 7fb4b73c6740 20 client.15465 trim_cache size 0 max 16384 2024-09-21T03:00:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.878+0000 7fb4b73c6740 1 Processor -- start 2024-09-21T03:00:19.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 -- start start 2024-09-21T03:00:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac495725e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 0x55ac49582e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ad6e0 con 0x55ac491c4350 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1aff60 con 0x55ac491a3670 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1add50 con 0x55ac4919b490 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39606/0 (socket says 172.21.15.16:39606) 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 -- 172.21.15.16:0/292685339 learned_addr learned my addr 172.21.15.16:0/292685339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 0x55ac49582e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac495725e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 -- 172.21.15.16:0/292685339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 msgr2=0x55ac495725e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac495725e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 -- 172.21.15.16:0/292685339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 msgr2=0x55ac49582e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 0x55ac49582e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 -- 172.21.15.16:0/292685339 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac49182f00 con 0x55ac491c4350 2024-09-21T03:00:19.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac495725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 0x55ac49582e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.879+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4012950 tx=0x7fb4a4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f03c8846eb3f8215 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb493fff640 1 -- 172.21.15.16:0/292685339 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a401d900 con 0x55ac491c4350 2024-09-21T03:00:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb493fff640 1 -- 172.21.15.16:0/292685339 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a401a410 con 0x55ac491c4350 2024-09-21T03:00:19.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb493fff640 1 -- 172.21.15.16:0/292685339 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a4026620 con 0x55ac491c4350 2024-09-21T03:00:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 -- 172.21.15.16:0/292685339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 msgr2=0x55ac49578d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4012950 tx=0x7fb4a4014000 comp rx=0 tx=0).stop 2024-09-21T03:00:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 -- 172.21.15.16:0/292685339 shutdown_connections 2024-09-21T03:00:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491c4350 0x55ac49578d10 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491a3670 0x55ac49582e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/292685339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac495725e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.880+0000 7fb4b73c6740 1 -- 172.21.15.16:0/292685339 >> 172.21.15.16:0/292685339 conn(0x55ac491b8740 msgr2=0x55ac4956c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- 172.21.15.16:0/292685339 shutdown_connections 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- 172.21.15.16:0/292685339 wait complete. 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- start start 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 Processor -- start 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4958a280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4958ad00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1ae420 con 0x55ac491a3670 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1af890 con 0x55ac491c4350 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb4ac1aeaf0 con 0x55ac4919b490 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4b73c6740 10 client.0 _ll_register_callbacks cb 0x7fb4a4009520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.881+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39616/0 (socket says 172.21.15.16:39616) 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/737071346 learned_addr learned my addr 172.21.15.16:0/737071346 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4958a280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4aad9d640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4958ad00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/737071346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 msgr2=0x55ac4958a280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4958a280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 msgr2=0x55ac4958ad00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4958ad00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 -- 172.21.15.16:0/737071346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac495f29f0 con 0x55ac491a3670 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4a9d9b640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55ac4956e2d0 tx=0x7fb4a0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62cf3edf9d1638b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb491ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb4a005c060 con 0x55ac491a3670 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb4a003baa0 con 0x55ac491a3670 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb4a401a410 con 0x55ac491a3670 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb4a003f070 con 0x55ac491a3670 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4b73c6740 10 client.15471 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:19.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.882+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ac495f3c00 con 0x55ac491a3670 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4a401a410 con 0x55ac491a3670 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb4a00320d0 con 0x55ac491a3670 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 10 client.15471 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 10 client.15471 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 20 client.15471 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49158080 0x55ac495f62d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 4 client.15471 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ac494df860 con 0x55ac49158080 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.883+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49158080 0x55ac495f62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.884+0000 7fb491ffb640 10 client.15471 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:19.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.884+0000 7fb4aa59c640 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49158080 0x55ac495f62d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3d428b1cc2a433a server_cookie=730ca4c239f8e5fb in_seq=0 out_seq=0 2024-09-21T03:00:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.885+0000 7fb491ffb640 1 -- 172.21.15.16:0/737071346 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ac494df860 con 0x55ac49158080 2024-09-21T03:00:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.885+0000 7fb491ffb640 10 client.15471 handle_command_reply: tid=0 2024-09-21T03:00:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.885+0000 7fb4b73c6740 1 client.15471 shutdown 2024-09-21T03:00:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 msgr2=0x55ac4958a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x55ac4956e2d0 tx=0x7fb4a0009780 comp rx=0 tx=0).stop 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 shutdown_connections 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ac49158080 0x55ac495f62d0 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ac491a3670 0x55ac4958a7c0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ac491c4350 0x55ac4958ad00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 --2- 172.21.15.16:0/737071346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ac4919b490 0x55ac4958a280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 >> 172.21.15.16:0/737071346 conn(0x55ac491b8740 msgr2=0x55ac49578ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb491ffb640 0 client.15471 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 shutdown_connections 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 1 -- 172.21.15.16:0/737071346 wait complete. 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b73c6740 20 client.15471 trim_cache size 0 max 16384 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:19.890 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "c2342f81-cbb6-415e-bcd5-bab15372fa32", 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 msgr2=0x7fb4940b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 0x7fb4940b8b00 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fb4a4005fd0 tx=0x7fb4a4009680 comp rx=0 tx=0).stop 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 msgr2=0x7fb4ac19a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.886+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7fb49c00d770 tx=0x7fb49c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 shutdown_connections 2024-09-21T03:00:19.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb4940b65f0 0x7fb4940b8b00 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb4ac10de10 0x7fb4ac19fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb4ac105290 0x7fb4ac19aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 --2- 172.21.15.16:0/154315861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb4ac104860 0x7fb4ac19a4e0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 >> 172.21.15.16:0/154315861 conn(0x7fb4ac079740 msgr2=0x7fb4ac1008a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 shutdown_connections 2024-09-21T03:00:19.892 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:19.887+0000 7fb4b0e52640 1 -- 172.21.15.16:0/154315861 wait complete. 2024-09-21T03:00:20.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:20 smithi016 ceph-mon[31253]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:20.638 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:20 smithi018 ceph-mon[34587]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:20.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:20 smithi123 ceph-mon[34645]: pgmap v320: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:20.898 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3629602969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa394108d00 msgr2=0x7fa39410f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3629602969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa394108d00 0x7fa39410f530 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fa38c009ca0 tx=0x7fa38c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3629602969 shutdown_connections 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3629602969 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa394108d00 0x7fa39410f530 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3629602969 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa3941083b0 0x7fa3941087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3629602969 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 0x7fa394102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3629602969 >> 172.21.15.16:0/3629602969 conn(0x7fa394079710 msgr2=0x7fa394079b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3629602969 shutdown_connections 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.418+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3629602969 wait complete. 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 Processor -- start 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 -- start start 2024-09-21T03:00:21.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 0x7fa394199350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394108d00 0x7fa39419eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941129c0 con 0x7fa3941083b0 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112840 con 0x7fa394102a70 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa39a9a8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa394112b40 con 0x7fa394108d00 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa38bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39640/0 (socket says 172.21.15.16:39640) 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa38bfff640 1 -- 172.21.15.16:0/3263392731 learned_addr learned my addr 172.21.15.16:0/3263392731 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.419+0000 7fa393fff640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 0x7fa394199350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa398f45640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394108d00 0x7fa39419eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 -- 172.21.15.16:0/3263392731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394108d00 msgr2=0x7fa39419eb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394108d00 0x7fa39419eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 -- 172.21.15.16:0/3263392731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 msgr2=0x7fa394199350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 0x7fa394199350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 -- 172.21.15.16:0/3263392731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa39419f200 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa38bfff640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d740 tx=0x7fa37c00db40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa391ffb640 1 -- 172.21.15.16:0/3263392731 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c017070 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa391ffb640 1 -- 172.21.15.16:0/3263392731 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa37c010460 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa39419f4f0 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa391ffb640 1 -- 172.21.15.16:0/3263392731 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa37c00f600 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.420+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3941a5d10 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 Processor -- start 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 -- start start 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa391ffb640 1 -- 172.21.15.16:0/3263392731 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7fa37c01a070 con 0x7fa3941083b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cde480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559142c52e50 con 0x55914290c1b0 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559142c42280 con 0x559142935070 2024-09-21T03:00:21.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559142c46090 con 0x559142914390 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.423+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39654/0 (socket says 172.21.15.16:39654) 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 -- 172.21.15.16:0/2118845452 learned_addr learned my addr 172.21.15.16:0/2118845452 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa398f45640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cde480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa38bfff640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 -- 172.21.15.16:0/2118845452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 msgr2=0x559142cdef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 -- 172.21.15.16:0/2118845452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 msgr2=0x559142cde480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cde480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 -- 172.21.15.16:0/2118845452 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591429a0650 con 0x55914290c1b0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa398f45640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cde480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa391ffb640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 0x7fa3800b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa393fff640 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa38400ad00 tx=0x7fa38400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b859029dd61465f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa398f45640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 0x7fa3800b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa391ffb640 1 -- 172.21.15.16:0/3263392731 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fa37c0e6f30 con 0x7fa3941083b0 2024-09-21T03:00:21.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa38a7fc640 1 -- 172.21.15.16:0/2118845452 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d070 con 0x55914290c1b0 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa38a7fc640 1 -- 172.21.15.16:0/2118845452 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384018af0 con 0x55914290c1b0 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.424+0000 7fa38a7fc640 1 -- 172.21.15.16:0/2118845452 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384010b30 con 0x55914290c1b0 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2118845452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 msgr2=0x559142cde9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa398f45640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 0x7fa3800b8bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006010 tx=0x7fa38c02f410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fa38400ad00 tx=0x7fa38400f790 comp rx=0 tx=0).stop 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2118845452 shutdown_connections 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cde9c0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2118845452 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cde480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2118845452 >> 172.21.15.16:0/2118845452 conn(0x559142929460 msgr2=0x559142cdc370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.425+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2118845452 shutdown_connections 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2118845452 wait complete. 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 -- start start 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 Processor -- start 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cea0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cf5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559142aa3700 con 0x55914290c1b0 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559142bdd110 con 0x559142935070 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.426+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559142cecc60 con 0x559142914390 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa3a0f1c740 10 client.0 _ll_register_callbacks cb 0x559142bf6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39662/0 (socket says 172.21.15.16:39662) 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 -- 172.21.15.16:0/2175530112 learned_addr learned my addr 172.21.15.16:0/2175530112 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa38bfff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cea0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa398f45640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cf5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 -- 172.21.15.16:0/2175530112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 msgr2=0x559142cea0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cea0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 -- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 msgr2=0x559142cf5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cf5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 -- 172.21.15.16:0/2175530112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559142c0ac40 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa38bfff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa398f45640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cf5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa393fff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa384010470 tx=0x7fa3840186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f15a6111071a77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa3737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d040 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fa384010870 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.427+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa384018af0 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa384031050 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 10 client.15483 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559142cf8980 con 0x55914290c1b0 2024-09-21T03:00:21.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384018af0 con 0x55914290c1b0 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fa384020030 con 0x55914290c1b0 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 10 client.15483 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 10 client.15483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 20 client.15483 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428f39f0 0x559142cfbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 4 client.15483 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:21.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.428+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559142c54da0 con 0x5591428f39f0 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.429+0000 7fa38bfff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428f39f0 0x559142cfbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.429+0000 7fa38bfff640 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428f39f0 0x559142cfbce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a103c199f72a3538 server_cookie=42d9a008afd0b95 in_seq=0 out_seq=0 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.429+0000 7fa3737fe640 10 client.15483 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3737fe640 1 -- 172.21.15.16:0/2175530112 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559142c54da0 con 0x5591428f39f0 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3737fe640 10 client.15483 handle_command_reply: tid=0 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 client.15483 shutdown 2024-09-21T03:00:21.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 msgr2=0x559142ce9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fa384010470 tx=0x7fa3840186f0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 shutdown_connections 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428f39f0 0x559142cfbce0 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cf5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cea0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/2175530112 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142ce9bb0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 >> 172.21.15.16:0/2175530112 conn(0x559142929460 msgr2=0x559142cd8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3737fe640 0 client.15483 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 shutdown_connections 2024-09-21T03:00:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/2175530112 wait complete. 2024-09-21T03:00:21.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.430+0000 7fa3a0f1c740 20 client.15483 trim_cache size 0 max 16384 2024-09-21T03:00:21.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.448+0000 7fa3a0f1c740 1 Processor -- start 2024-09-21T03:00:21.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 -- start start 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdfe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cd8e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ab520 con 0x55914290c1b0 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941adda0 con 0x559142935070 2024-09-21T03:00:21.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941abb90 con 0x559142914390 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39672/0 (socket says 172.21.15.16:39672) 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 -- 172.21.15.16:0/3675517228 learned_addr learned my addr 172.21.15.16:0/3675517228 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa398f45640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cd8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa38bfff640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdfe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 -- 172.21.15.16:0/3675517228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 msgr2=0x559142cdfe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdfe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 -- 172.21.15.16:0/3675517228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 msgr2=0x559142cd8e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cd8e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa393fff640 1 -- 172.21.15.16:0/3675517228 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559142ce12e0 con 0x55914290c1b0 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.449+0000 7fa38bfff640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.450+0000 7fa393fff640 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa38400ac50 tx=0x7fa38400a2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c64aa41783a8346 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.450+0000 7fa389ffb640 1 -- 172.21.15.16:0/3675517228 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d040 con 0x55914290c1b0 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.450+0000 7fa389ffb640 1 -- 172.21.15.16:0/3675517228 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384028070 con 0x55914290c1b0 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.450+0000 7fa389ffb640 1 -- 172.21.15.16:0/3675517228 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa384018760 con 0x55914290c1b0 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/3675517228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 msgr2=0x559142cea130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fa38400ac50 tx=0x7fa38400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/3675517228 shutdown_connections 2024-09-21T03:00:21.467 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142935070 0x559142cd8e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142914390 0x559142cdfe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/3675517228 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cea130 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/3675517228 >> 172.21.15.16:0/3675517228 conn(0x559142929460 msgr2=0x559142cd8700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/3675517228 shutdown_connections 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/3675517228 wait complete. 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 -- start start 2024-09-21T03:00:21.468 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.451+0000 7fa3a0f1c740 1 Processor -- start 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142914390 0x559142cfc8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142935070 0x559142cfcde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ac260 con 0x55914290c1b0 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ad6d0 con 0x559142914390 2024-09-21T03:00:21.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa3941ac930 con 0x559142935070 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3a0f1c740 10 client.0 _ll_register_callbacks cb 0x7fa38401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39674/0 (socket says 172.21.15.16:39674) 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 -- 172.21.15.16:0/1541594541 learned_addr learned my addr 172.21.15.16:0/1541594541 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa38bfff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142914390 0x559142cfc8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa398f45640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142935070 0x559142cfcde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 -- 172.21.15.16:0/1541594541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142935070 msgr2=0x559142cfcde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142935070 0x559142cfcde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 -- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142914390 msgr2=0x559142cfc8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142914390 0x559142cfc8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 -- 172.21.15.16:0/1541594541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559142cd5580 con 0x55914290c1b0 2024-09-21T03:00:21.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa393fff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa3840057d0 tx=0x7fa384010640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30779bb3b7409166 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:21.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.452+0000 7fa3737fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:21.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa38401d040 con 0x55914290c1b0 2024-09-21T03:00:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fa384010ba0 con 0x55914290c1b0 2024-09-21T03:00:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa384028070 con 0x55914290c1b0 2024-09-21T03:00:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fa384010dc0 con 0x55914290c1b0 2024-09-21T03:00:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 10 client.15489 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:21.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559142cd9780 con 0x55914290c1b0 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa384028070 con 0x55914290c1b0 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fa384045020 con 0x55914290c1b0 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 10 client.15489 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 10 client.15489 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 20 client.15489 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428c8da0 0x559142d68510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:21.475 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 4 client.15489 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.453+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559142c50850 con 0x5591428c8da0 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa38bfff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428c8da0 0x559142d68510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa38bfff640 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428c8da0 0x559142d68510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e8c6aa50f2ba075 server_cookie=67ff692d42a820ed in_seq=0 out_seq=0 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3737fe640 10 client.15489 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3737fe640 1 -- 172.21.15.16:0/1541594541 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+854 (crc 0 0 0) 0x559142c50850 con 0x5591428c8da0 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3737fe640 10 client.15489 handle_command_reply: tid=0 2024-09-21T03:00:21.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 client.15489 shutdown 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 msgr2=0x559142cff3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fa3840057d0 tx=0x7fa384010640 comp rx=0 tx=0).stop 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 shutdown_connections 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5591428c8da0 0x559142d68510 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559142935070 0x559142cfcde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559142914390 0x559142cfc8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.477 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 --2- 172.21.15.16:0/1541594541 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55914290c1b0 0x559142cff3d0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 >> 172.21.15.16:0/1541594541 conn(0x559142929460 msgr2=0x559142cdfc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.454+0000 7fa3737fe640 0 client.15489 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 shutdown_connections 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa3a0f1c740 1 -- 172.21.15.16:0/1541594541 wait complete. 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa3a0f1c740 20 client.15489 trim_cache size 0 max 16384 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:21.478 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "c2342f81-cbb6-415e-bcd5-bab15372fa32", 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:00:21.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:21.480 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 msgr2=0x7fa3800b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 0x7fa3800b8bb0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fa38c006010 tx=0x7fa38c02f410 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 msgr2=0x7fa3941998f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fa37c00d740 tx=0x7fa37c00db40 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 shutdown_connections 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa3800b66a0 0x7fa3800b8bb0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa3941083b0 0x7fa3941998f0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa394108d00 0x7fa39419eb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 --2- 172.21.15.16:0/3263392731 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa394102a70 0x7fa394199350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 >> 172.21.15.16:0/3263392731 conn(0x7fa394079710 msgr2=0x7fa394106400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 shutdown_connections 2024-09-21T03:00:21.481 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:21.455+0000 7fa39a9a8640 1 -- 172.21.15.16:0/3263392731 wait complete. 2024-09-21T03:00:21.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:21 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:00:21.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:21 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:21.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:21 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:00:21.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:21 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:21.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:21 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:00:21.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:21 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:21.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:21 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:00:21.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:21 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:21.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:21 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:00:21.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:21 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:21.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:21 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:00:21.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:21 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:00:22.194 INFO:tasks.cephfs.filesystem:scrub status for tag:c2342f81-cbb6-415e-bcd5-bab15372fa32 - {'path': '/', 'tag': 'c2342f81-cbb6-415e-bcd5-bab15372fa32', 'options': 'recursive,force'} 2024-09-21T03:00:22.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:22 smithi016 ceph-mon[31253]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:22 smithi018 ceph-mon[34587]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:22 smithi123 ceph-mon[34645]: pgmap v321: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:23.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:23 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:00:23.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:23 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:00:23.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:23 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:00:24.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:24 smithi018 ceph-mon[34587]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:24.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:24 smithi123 ceph-mon[34645]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:24.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:24 smithi016 ceph-mon[31253]: pgmap v322: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 175 KiB/s wr, 48 op/s 2024-09-21T03:00:26.567 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:26 smithi016 ceph-mon[31253]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:26.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:26 smithi018 ceph-mon[34587]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:26 smithi123 ceph-mon[34645]: pgmap v323: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 175 KiB/s wr, 49 op/s 2024-09-21T03:00:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:28 smithi018 ceph-mon[34587]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:28.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:28 smithi123 ceph-mon[34645]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:28.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:28 smithi016 ceph-mon[31253]: pgmap v324: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:30.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:30 smithi123 ceph-mon[34645]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:30.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:30 smithi018 ceph-mon[34587]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:30 smithi016 ceph-mon[31253]: pgmap v325: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:00:32.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:32 smithi016 ceph-mon[31253]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-21T03:00:32.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:32 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:32 smithi018 ceph-mon[34587]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-21T03:00:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:32 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:32 smithi123 ceph-mon[34645]: pgmap v326: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.4 KiB/s rd, 2 op/s 2024-09-21T03:00:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:32 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:00:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:34.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:34.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: pgmap v327: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:00:34.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:00:36.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:36 smithi016 ceph-mon[31253]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-21T03:00:36.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:36 smithi018 ceph-mon[34587]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-21T03:00:36.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:36 smithi123 ceph-mon[34645]: pgmap v328: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 2 op/s 2024-09-21T03:00:38.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:38 smithi016 ceph-mon[31253]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:38.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:38 smithi018 ceph-mon[34587]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:38.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:38 smithi123 ceph-mon[34645]: pgmap v329: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:40.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:40 smithi016 ceph-mon[31253]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:40.522 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:40 smithi123 ceph-mon[34645]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:40.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:40 smithi018 ceph-mon[34587]: pgmap v330: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:42.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:42 smithi016 ceph-mon[31253]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:42.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:42 smithi018 ceph-mon[34587]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:42.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:42 smithi123 ceph-mon[34645]: pgmap v331: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:44.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:44 smithi016 ceph-mon[31253]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:44.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:44 smithi018 ceph-mon[34587]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:44.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:44 smithi123 ceph-mon[34645]: pgmap v332: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:46.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:46 smithi016 ceph-mon[31253]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:46.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:46 smithi018 ceph-mon[34587]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:46.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:46 smithi123 ceph-mon[34645]: pgmap v333: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:00:48.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:48 smithi016 ceph-mon[31253]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:48.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:48 smithi018 ceph-mon[34587]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:48 smithi123 ceph-mon[34645]: pgmap v334: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:50.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:50 smithi016 ceph-mon[31253]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:50.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:50 smithi018 ceph-mon[34587]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:50.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:50 smithi123 ceph-mon[34645]: pgmap v335: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:00:52.195 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:00:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:52.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:52 smithi016 ceph-mon[31253]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:52 smithi018 ceph-mon[34587]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:52.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:52 smithi123 ceph-mon[34645]: pgmap v336: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 -- 172.21.15.16:0/1207627297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 msgr2=0x7f727410d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 --2- 172.21.15.16:0/1207627297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f727410d910 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f7268009900 tx=0x7f726802a410 comp rx=0 tx=0).stop 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 -- 172.21.15.16:0/1207627297 shutdown_connections 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 --2- 172.21.15.16:0/1207627297 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f727410d910 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 --2- 172.21.15.16:0/1207627297 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 0x7f7274110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 --2- 172.21.15.16:0/1207627297 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f7274104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 -- 172.21.15.16:0/1207627297 >> 172.21.15.16:0/1207627297 conn(0x7f7274079740 msgr2=0x7f7274079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 -- 172.21.15.16:0/1207627297 shutdown_connections 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.001+0000 7f727a7f8640 1 -- 172.21.15.16:0/1207627297 wait complete. 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.002+0000 7f727a7f8640 1 Processor -- start 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.002+0000 7f727a7f8640 1 -- start start 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.002+0000 7f727a7f8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f72741027b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.002+0000 7f727a7f8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.002+0000 7f727a7f8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 0x7f7274103230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f727a7f8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7274113a90 con 0x7f72741052d0 2024-09-21T03:00:53.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f727a7f8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7274113910 con 0x7f72741048a0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f727a7f8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7274113c10 con 0x7f727410de50 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43100/0 (socket says 172.21.15.16:43100) 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 -- 172.21.15.16:0/660501873 learned_addr learned my addr 172.21.15.16:0/660501873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f7278d95640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 0x7f7274103230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f7273fff640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f72741027b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 -- 172.21.15.16:0/660501873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 msgr2=0x7f7274103230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 0x7f7274103230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 -- 172.21.15.16:0/660501873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 msgr2=0x7f72741027b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f72741027b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f72737fe640 1 -- 172.21.15.16:0/660501873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72741a8720 con 0x7f72741052d0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.003+0000 7f7273fff640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f72741027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.004+0000 7f72737fe640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7268006010 tx=0x7f7268004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.004+0000 7f72717fa640 1 -- 172.21.15.16:0/660501873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726803f030 con 0x7f72741052d0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.004+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72741a89b0 con 0x7f72741052d0 2024-09-21T03:00:53.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.004+0000 7f72717fa640 1 -- 172.21.15.16:0/660501873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7268003aa0 con 0x7f72741052d0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.004+0000 7f72717fa640 1 -- 172.21.15.16:0/660501873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7268035070 con 0x7f72741052d0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.005+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72741a8ee0 con 0x7f72741052d0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.007+0000 7f72717fa640 1 -- 172.21.15.16:0/660501873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f7268042030 con 0x7f72741052d0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.008+0000 7f72717fa640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 0x7f725c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.008+0000 7f7273fff640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 0x7f725c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.008+0000 7f72717fa640 1 -- 172.21.15.16:0/660501873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f7268107ab0 con 0x7f72741052d0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 Processor -- start 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 -- start start 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7273fff640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 0x7f725c0b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7274104200 tx=0x7f726400b6b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a11e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a123a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d16497fe50 con 0x55d164641390 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d16496f280 con 0x55d164662070 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7280d6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d164973090 con 0x55d1646391b0 2024-09-21T03:00:53.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.009+0000 7f7278d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43104/0 (socket says 172.21.15.16:43104) 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 -- 172.21.15.16:0/4037979186 learned_addr learned my addr 172.21.15.16:0/4037979186 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7273fff640 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a11e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f72737fe640 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a123a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 -- 172.21.15.16:0/4037979186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 msgr2=0x55d164a123a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a123a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 -- 172.21.15.16:0/4037979186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 msgr2=0x55d164a11e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a11e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 -- 172.21.15.16:0/4037979186 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1646cd650 con 0x55d164641390 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.010+0000 7f7278d95640 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f726c009f50 tx=0x7f726c0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1582d33ca9e95cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.011+0000 7f725a7fc640 1 -- 172.21.15.16:0/4037979186 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c004d20 con 0x55d164641390 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.011+0000 7f725a7fc640 1 -- 172.21.15.16:0/4037979186 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c004050 con 0x55d164641390 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.011+0000 7f725a7fc640 1 -- 172.21.15.16:0/4037979186 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f726c005280 con 0x55d164641390 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 -- 172.21.15.16:0/4037979186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 msgr2=0x55d164a128e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f726c009f50 tx=0x7f726c0118d0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 -- 172.21.15.16:0/4037979186 shutdown_connections 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a128e0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a123a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 --2- 172.21.15.16:0/4037979186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a11e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 -- 172.21.15.16:0/4037979186 >> 172.21.15.16:0/4037979186 conn(0x55d164656460 msgr2=0x55d164a0fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 -- 172.21.15.16:0/4037979186 shutdown_connections 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.012+0000 7f7280d6c740 1 -- 172.21.15.16:0/4037979186 wait complete. 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.013+0000 7f7280d6c740 1 -- start start 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.013+0000 7f7280d6c740 1 Processor -- start 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.013+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a1d550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.013+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a29930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43108/0 (socket says 172.21.15.16:43108) 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 -- 172.21.15.16:0/1628094926 learned_addr learned my addr 172.21.15.16:0/1628094926 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d1647d0700 con 0x55d164641390 2024-09-21T03:00:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d16490a110 con 0x55d164662070 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d164a20640 con 0x55d1646391b0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7273fff640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a1d550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7280d6c740 10 client.0 _ll_register_callbacks cb 0x55d164923990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7278d95640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a29930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 -- 172.21.15.16:0/1628094926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 msgr2=0x55d164a1d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a1d550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 -- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 msgr2=0x55d164a29930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a29930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d164937c40 con 0x55d164641390 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f7278d95640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a29930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72737fe640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7268031440 tx=0x7f7268000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea5d2e236da44a8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.014+0000 7f72537fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680d2020 con 0x55d164641390 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f726c004050 con 0x55d164641390 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f72680d6f70 con 0x55d164641390 2024-09-21T03:00:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7268049040 con 0x55d164641390 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f7280d6c740 10 client.15501 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d164a2c380 con 0x55d164641390 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.015+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f726c004050 con 0x55d164641390 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f72680e3b90 con 0x55d164641390 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 10 client.15501 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 10 client.15501 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 20 client.15501 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d1646209f0 0x55d164a2fa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 4 client.15501 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d164981da0 con 0x55d1646209f0 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.016+0000 7f7273fff640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d1646209f0 0x55d164a2fa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.017+0000 7f7273fff640 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d1646209f0 0x55d164a2fa60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=13561dcda8251b63 server_cookie=9b63f35b106e543d in_seq=0 out_seq=0 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.017+0000 7f72537fe640 10 client.15501 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f72537fe640 1 -- 172.21.15.16:0/1628094926 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d164981da0 con 0x55d1646209f0 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f72537fe640 10 client.15501 handle_command_reply: tid=0 2024-09-21T03:00:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 client.15501 shutdown 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 msgr2=0x55d164a1da90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f7268031440 tx=0x7f7268000bc0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 shutdown_connections 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d1646209f0 0x55d164a2fa60 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164641390 0x55d164a1da90 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164662070 0x55d164a29930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 --2- 172.21.15.16:0/1628094926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a1d550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 >> 172.21.15.16:0/1628094926 conn(0x55d164656460 msgr2=0x55d164a05b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f72537fe640 0 client.15501 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 shutdown_connections 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 1 -- 172.21.15.16:0/1628094926 wait complete. 2024-09-21T03:00:53.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.018+0000 7f7280d6c740 20 client.15501 trim_cache size 0 max 16384 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 Processor -- start 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 -- start start 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a29830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a1a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72741ab530 con 0x55d164656460 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f72741add90 con 0x55d164641390 2024-09-21T03:00:53.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.036+0000 7f7280d6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72741abb80 con 0x55d1646391b0 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43116/0 (socket says 172.21.15.16:43116) 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 -- 172.21.15.16:0/3367846129 learned_addr learned my addr 172.21.15.16:0/3367846129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7278d95640 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a1a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f72737fe640 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a29830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 -- 172.21.15.16:0/3367846129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 msgr2=0x55d164a29830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a29830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 -- 172.21.15.16:0/3367846129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 msgr2=0x55d164a1a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a1a740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 -- 172.21.15.16:0/3367846129 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d164a18190 con 0x55d164656460 2024-09-21T03:00:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7273fff640 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7264005fd0 tx=0x7f7264012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fb894883f5e4227 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.037+0000 7f7259ffb640 1 -- 172.21.15.16:0/3367846129 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264021190 con 0x55d164656460 2024-09-21T03:00:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.038+0000 7f7259ffb640 1 -- 172.21.15.16:0/3367846129 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264021d50 con 0x55d164656460 2024-09-21T03:00:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.038+0000 7f7259ffb640 1 -- 172.21.15.16:0/3367846129 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7264031990 con 0x55d164656460 2024-09-21T03:00:53.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 -- 172.21.15.16:0/3367846129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 msgr2=0x55d164a135a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7f7264005fd0 tx=0x7f7264012000 comp rx=0 tx=0).stop 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 -- 172.21.15.16:0/3367846129 shutdown_connections 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a1a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d1646391b0 0x55d164a29830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 --2- 172.21.15.16:0/3367846129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d164656460 0x55d164a135a0 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 -- 172.21.15.16:0/3367846129 >> 172.21.15.16:0/3367846129 conn(0x55d164a02730 msgr2=0x55d164a12e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 -- 172.21.15.16:0/3367846129 shutdown_connections 2024-09-21T03:00:53.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.039+0000 7f7280d6c740 1 -- 172.21.15.16:0/3367846129 wait complete. 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 -- start start 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 Processor -- start 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a26130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d164656460 0x55d164a20890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f72741ac250 con 0x55d1646391b0 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f72741ad6c0 con 0x55d164641390 2024-09-21T03:00:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7273fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.040+0000 7f7280d6c740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f72741ac920 con 0x55d164656460 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f7273fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43128/0 (socket says 172.21.15.16:43128) 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f7273fff640 1 -- 172.21.15.16:0/155178468 learned_addr learned my addr 172.21.15.16:0/155178468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f7280d6c740 10 client.0 _ll_register_callbacks cb 0x7f7268005ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a26130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f7278d95640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d164656460 0x55d164a20890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 -- 172.21.15.16:0/155178468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d164656460 msgr2=0x55d164a20890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d164656460 0x55d164a20890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 -- 172.21.15.16:0/155178468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 msgr2=0x55d164a25b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 -- 172.21.15.16:0/155178468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d164a21020 con 0x55d164641390 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f7273fff640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72737fe640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a26130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72680cd970 tx=0x7f72680d7180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab37af2cfcbf66d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.041+0000 7f72537fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:00:53.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f72680d2040 con 0x55d164641390 2024-09-21T03:00:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7264021d50 con 0x55d164641390 2024-09-21T03:00:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f72680037f0 con 0x55d164641390 2024-09-21T03:00:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f72680ec070 con 0x55d164641390 2024-09-21T03:00:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 10 client.25214 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:53.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d164a9ae90 con 0x55d164641390 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7264021d50 con 0x55d164641390 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f72680ccc40 con 0x55d164641390 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 10 client.25214 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 10 client.25214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 20 client.25214 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d164662070 0x55d164a9d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 4 client.25214 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.042+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d16497d850 con 0x55d164662070 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f7273fff640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d164662070 0x55d164a9d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f72537fe640 10 client.25214 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:53.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f7273fff640 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d164662070 0x55d164a9d3d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b0675948e092ec3d server_cookie=bd343a586af6f659 in_seq=0 out_seq=0 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f72537fe640 1 -- 172.21.15.16:0/155178468 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+658 (crc 0 0 0) 0x55d16497d850 con 0x55d164662070 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f72537fe640 10 client.25214 handle_command_reply: tid=0 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.043+0000 7f7280d6c740 1 client.25214 shutdown 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 msgr2=0x55d164a26130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a26130 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f72680cd970 tx=0x7f72680d7180 comp rx=0 tx=0).stop 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 shutdown_connections 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d164662070 0x55d164a9d3d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d164656460 0x55d164a20890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d164641390 0x55d164a26130 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 --2- 172.21.15.16:0/155178468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d1646391b0 0x55d164a25b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 >> 172.21.15.16:0/155178468 conn(0x55d164a02730 msgr2=0x55d164a12e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f72537fe640 0 client.25214 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 shutdown_connections 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 1 -- 172.21.15.16:0/155178468 wait complete. 2024-09-21T03:00:53.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.044+0000 7f7280d6c740 20 client.25214 trim_cache size 0 max 16384 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:00:53.049 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:00:53.050 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 msgr2=0x7f725c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 0x7f725c0b8b00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7274104200 tx=0x7f726400b6b0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 msgr2=0x7f7274102cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f7268006010 tx=0x7f7268004060 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 shutdown_connections 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f725c0b65f0 0x7f725c0b8b00 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f72741052d0 0x7f7274102cf0 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f727410de50 0x7f7274103230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 --2- 172.21.15.16:0/660501873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f72741048a0 0x7f72741027b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 >> 172.21.15.16:0/660501873 conn(0x7f7274079740 msgr2=0x7f72741068d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 shutdown_connections 2024-09-21T03:00:53.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:53.045+0000 7f727a7f8640 1 -- 172.21.15.16:0/660501873 wait complete. 2024-09-21T03:00:53.890 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:00:53.890 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:00:54.217 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:54.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:54 smithi016 ceph-mon[31253]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:54.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:54 smithi018 ceph-mon[34587]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:54.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:54 smithi123 ceph-mon[34645]: pgmap v337: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 -- 172.21.15.16:0/4101171944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 msgr2=0x7f5fac10d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/4101171944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac10d8d0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c009900 tx=0x7f5f9c02a410 comp rx=0 tx=0).stop 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 -- 172.21.15.16:0/4101171944 shutdown_connections 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/4101171944 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac10d8d0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/4101171944 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 0x7f5fac110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/4101171944 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 0x7f5fac104e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 -- 172.21.15.16:0/4101171944 >> 172.21.15.16:0/4101171944 conn(0x7f5fac0797b0 msgr2=0x7f5fac079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 -- 172.21.15.16:0/4101171944 shutdown_connections 2024-09-21T03:00:54.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.763+0000 7f5fb37ec640 1 -- 172.21.15.16:0/4101171944 wait complete. 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.764+0000 7f5fb37ec640 1 Processor -- start 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.764+0000 7f5fb37ec640 1 -- start start 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.764+0000 7f5fb37ec640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 0x7f5fac198320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.764+0000 7f5fb37ec640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb37ec640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 0x7f5fac19db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb37ec640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac104220 con 0x7f5fac105380 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb37ec640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1040a0 con 0x7f5fac104a30 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb37ec640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5fac1043a0 con 0x7f5fac10de60 2024-09-21T03:00:54.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60412/0 (socket says 172.21.15.16:60412) 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 -- 172.21.15.16:0/120445708 learned_addr learned my addr 172.21.15.16:0/120445708 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb1588640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 0x7f5fac198320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb1d89640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 0x7f5fac19db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 -- 172.21.15.16:0/120445708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 msgr2=0x7f5fac19db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 0x7f5fac19db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 -- 172.21.15.16:0/120445708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 msgr2=0x7f5fac198320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 0x7f5fac198320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.765+0000 7f5fb0d87640 1 -- 172.21.15.16:0/120445708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5fac19e290 con 0x7f5fac105380 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.766+0000 7f5fb0d87640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006010 tx=0x7f5f9c004060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.766+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c03f030 con 0x7f5fac105380 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.766+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5f9c003aa0 con 0x7f5fac105380 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.766+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5f9c035070 con 0x7f5fac105380 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.766+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5fac19e520 con 0x7f5fac105380 2024-09-21T03:00:54.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.767+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5fac1a4ce0 con 0x7f5fac105380 2024-09-21T03:00:54.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.769+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5fac104e40 con 0x7f5fac105380 2024-09-21T03:00:54.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.772+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f5f9c042030 con 0x7f5fac105380 2024-09-21T03:00:54.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.773+0000 7f5fa27fc640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 0x7f5f980b8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.773+0000 7f5fb1588640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 0x7f5f980b8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.774+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5f9c107e40 con 0x7f5fac105380 2024-09-21T03:00:54.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.774+0000 7f5fb1588640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 0x7f5f980b8c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8005d00 tx=0x7f5fa800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:54.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:54.776+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5f9c0ce4f0 con 0x7f5fac105380 2024-09-21T03:00:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.018+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5fac108f40 con 0x7f5fac105380 2024-09-21T03:00:55.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.019+0000 7f5fa27fc640 1 -- 172.21.15.16:0/120445708 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f5f9c0d28d0 con 0x7f5fac105380 2024-09-21T03:00:55.021 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:00:55.021 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:00:55.021 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:00:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 msgr2=0x7f5f980b8c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 0x7f5f980b8c30 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f5fa8005d00 tx=0x7f5fa800b410 comp rx=0 tx=0).stop 2024-09-21T03:00:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 msgr2=0x7f5fac1988c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:55.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f5f9c006010 tx=0x7f5f9c004060 comp rx=0 tx=0).stop 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 shutdown_connections 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5f980b6720 0x7f5f980b8c30 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5fac105380 0x7f5fac1988c0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac10de60 0x7f5fac19db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 --2- 172.21.15.16:0/120445708 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5fac104a30 0x7f5fac198320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 >> 172.21.15.16:0/120445708 conn(0x7f5fac0797b0 msgr2=0x7f5fac0ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 shutdown_connections 2024-09-21T03:00:55.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:55.022+0000 7f5fb37ec640 1 -- 172.21.15.16:0/120445708 wait complete. 2024-09-21T03:00:55.351 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:55 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/120445708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:55.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:55 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/120445708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:55.686 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:00:55.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:55 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/120445708' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:00:55.971 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:56.270 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:56 smithi016 ceph-mon[31253]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 -- 172.21.15.16:0/4053213252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0106bd0 msgr2=0x7f3da0110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 --2- 172.21.15.16:0/4053213252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0106bd0 0x7f3da0110660 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f3d98009ca0 tx=0x7f3d9802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 -- 172.21.15.16:0/4053213252 shutdown_connections 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 --2- 172.21.15.16:0/4053213252 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0106bd0 0x7f3da0110660 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 --2- 172.21.15.16:0/4053213252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106280 0x7f3da0106690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 --2- 172.21.15.16:0/4053213252 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0102030 0x7f3da0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.479+0000 7f3da76de640 1 -- 172.21.15.16:0/4053213252 >> 172.21.15.16:0/4053213252 conn(0x7f3da0077c60 msgr2=0x7f3da00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.480+0000 7f3da76de640 1 -- 172.21.15.16:0/4053213252 shutdown_connections 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.480+0000 7f3da76de640 1 -- 172.21.15.16:0/4053213252 wait complete. 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.480+0000 7f3da76de640 1 Processor -- start 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.480+0000 7f3da76de640 1 -- start start 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0106280 0x7f3da019aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106bd0 0x7f3da019fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0113ac0 con 0x7f3da0102030 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0113940 con 0x7f3da0106280 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da76de640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3da0113c40 con 0x7f3da0106bd0 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60434/0 (socket says 172.21.15.16:60434) 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 -- 172.21.15.16:0/1865215340 learned_addr learned my addr 172.21.15.16:0/1865215340 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da4c79640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0106280 0x7f3da019aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106bd0 0x7f3da019fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 -- 172.21.15.16:0/1865215340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106bd0 msgr2=0x7f3da019fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106bd0 0x7f3da019fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 -- 172.21.15.16:0/1865215340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0106280 msgr2=0x7f3da019aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0106280 0x7f3da019aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 -- 172.21.15.16:0/1865215340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3da01a0440 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.481+0000 7f3da547a640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e9c0 tx=0x7f3d9c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.482+0000 7f3d967fc640 1 -- 172.21.15.16:0/1865215340 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c00cde0 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.482+0000 7f3d967fc640 1 -- 172.21.15.16:0/1865215340 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d9c0108e0 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.482+0000 7f3d967fc640 1 -- 172.21.15.16:0/1865215340 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c018960 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.482+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3da01a0730 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.482+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3da0102eb0 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.485+0000 7f3d967fc640 1 -- 172.21.15.16:0/1865215340 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f3d9c01f020 con 0x7f3da0102030 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3dadc52740 1 Processor -- start 2024-09-21T03:00:56.492 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3dadc52740 1 -- start start 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3d967fc640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 0x7f3d8c0b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 0x562b3b57c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3d967fc640 1 -- 172.21.15.16:0/1865215340 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f3d9c0e79e0 con 0x7f3da0102030 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.486+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 0x7f3d8c0b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da547a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 0x562b3b57c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da547a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 0x562b3b57c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44758/0 (socket says 172.21.15.16:44758) 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60448/0 (socket says 172.21.15.16:60448) 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1b4390 0x562b3b57cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da547a640 1 -- 172.21.15.16:0/670133586 learned_addr learned my addr 172.21.15.16:0/670133586 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b3b4f2ea0 con 0x562b3b1ac1b0 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b3b4e22d0 con 0x562b3b1b4390 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3b4e60e0 con 0x562b3b1d5070 2024-09-21T03:00:56.493 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1b4390 0x562b3b57cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 -- 172.21.15.16:0/670133586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 msgr2=0x562b3b57c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 0x562b3b57c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 -- 172.21.15.16:0/670133586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1b4390 msgr2=0x562b3b57cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1b4390 0x562b3b57cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.487+0000 7f3da4c79640 1 -- 172.21.15.16:0/670133586 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3b240650 con 0x562b3b1ac1b0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.488+0000 7f3da4c79640 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3da0075820 tx=0x7f3d90009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79a8d20c8e4b658 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.488+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 0x7f3d8c0b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3d9802f500 tx=0x7f3d98031bc0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.488+0000 7f3d8b7fe640 1 -- 172.21.15.16:0/670133586 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d90011d70 con 0x562b3b1ac1b0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.488+0000 7f3d8b7fe640 1 -- 172.21.15.16:0/670133586 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90012900 con 0x562b3b1ac1b0 2024-09-21T03:00:56.494 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.488+0000 7f3d8b7fe640 1 -- 172.21.15.16:0/670133586 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9000f840 con 0x562b3b1ac1b0 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 msgr2=0x562b3b57c800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7f3da0075820 tx=0x7f3d90009f50 comp rx=0 tx=0).stop 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 shutdown_connections 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1ac1b0 0x562b3b57c800 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1b4390 0x562b3b57cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 --2- 172.21.15.16:0/670133586 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1d5070 0x562b3b57c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 >> 172.21.15.16:0/670133586 conn(0x562b3b1c9460 msgr2=0x562b3b57a1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 shutdown_connections 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.489+0000 7f3dadc52740 1 -- 172.21.15.16:0/670133586 wait complete. 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 -- start start 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 Processor -- start 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1b4390 0x562b3b587ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1d5070 0x562b3b593d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562b3b343700 con 0x562b3b1d5070 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562b3b47d160 con 0x562b3b1ac1b0 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.490+0000 7f3dadc52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562b3b58aaa0 con 0x562b3b1b4390 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:46916/0 (socket says 172.21.15.16:46916) 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 -- 172.21.15.16:0/2974729575 learned_addr learned my addr 172.21.15.16:0/2974729575 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3dadc52740 10 client.0 _ll_register_callbacks cb 0x562b3b4969e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1d5070 0x562b3b593d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da4c79640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1b4390 0x562b3b587ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 -- 172.21.15.16:0/2974729575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1b4390 msgr2=0x562b3b587ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1b4390 0x562b3b587ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 -- 172.21.15.16:0/2974729575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1d5070 msgr2=0x562b3b593d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1d5070 0x562b3b593d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 -- 172.21.15.16:0/2974729575 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3b4aac90 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3da547a640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad8a0 tx=0x7f3d9c0e5710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a821d337e8f914b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.491+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0a9080 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d90012900 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f3d9c010430 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3d9c0ad420 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3dadc52740 10 client.25232 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b3b5967c0 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.492+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d90012900 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f3d9c018b00 con 0x562b3b1ac1b0 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 10 client.25232 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 10 client.25232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 20 client.25232 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1939f0 0x562b3b599d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 4 client.25232 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b3b4f4df0 con 0x562b3b1939f0 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.493+0000 7f3da4c79640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1939f0 0x562b3b599d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.494+0000 7f3d88ff9640 10 client.25232 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.494+0000 7f3da4c79640 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1939f0 0x562b3b599d50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22bc50cb67c7e29b server_cookie=af91ca53dab03990 in_seq=0 out_seq=0 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.494+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2974729575 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562b3b4f4df0 con 0x562b3b1939f0 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.494+0000 7f3d88ff9640 10 client.25232 handle_command_reply: tid=0 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.494+0000 7f3dadc52740 1 client.25232 shutdown 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 msgr2=0x562b3b587990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad8a0 tx=0x7f3d9c0e5710 comp rx=0 tx=0).stop 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 shutdown_connections 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1939f0 0x562b3b599d50 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1d5070 0x562b3b593d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3d88ff9640 0 client.25232 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1b4390 0x562b3b587ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2974729575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b587990 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 >> 172.21.15.16:0/2974729575 conn(0x562b3b1c9460 msgr2=0x562b3b57f450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 shutdown_connections 2024-09-21T03:00:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.495+0000 7f3dadc52740 1 -- 172.21.15.16:0/2974729575 wait complete. 2024-09-21T03:00:56.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.496+0000 7f3dadc52740 20 client.25232 trim_cache size 0 max 16384 2024-09-21T03:00:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.522+0000 7f3dadc52740 1 Processor -- start 2024-09-21T03:00:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.522+0000 7f3dadc52740 1 -- start start 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.522+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 0x562b3b57ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.522+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1ac1b0 0x562b3b589870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.522+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b593c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da5c7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b593c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3dadc52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ab4f0 con 0x562b3b1b4390 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3dadc52740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01add70 con 0x562b3b1939f0 2024-09-21T03:00:56.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3dadc52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01abb60 con 0x562b3b1ac1b0 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 0x562b3b57ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da5c7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b593c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60452/0 (socket says 172.21.15.16:60452) 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da5c7b640 1 -- 172.21.15.16:0/1346234802 learned_addr learned my addr 172.21.15.16:0/1346234802 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da4c79640 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1ac1b0 0x562b3b589870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 -- 172.21.15.16:0/1346234802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1ac1b0 msgr2=0x562b3b589870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1ac1b0 0x562b3b589870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 -- 172.21.15.16:0/1346234802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 msgr2=0x562b3b593c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b593c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 -- 172.21.15.16:0/1346234802 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b3b588c50 con 0x562b3b1939f0 2024-09-21T03:00:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3da547a640 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 0x562b3b57ddc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad5c0 tx=0x7f3d9c018420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f49dd40af924b93b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.523+0000 7f3d8affd640 1 -- 172.21.15.16:0/1346234802 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0ad080 con 0x562b3b1939f0 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.524+0000 7f3d8affd640 1 -- 172.21.15.16:0/1346234802 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d9c0ae010 con 0x562b3b1939f0 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.524+0000 7f3d8affd640 1 -- 172.21.15.16:0/1346234802 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d9c0acb70 con 0x562b3b1939f0 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.524+0000 7f3dadc52740 1 -- 172.21.15.16:0/1346234802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 msgr2=0x562b3b57ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.524+0000 7f3dadc52740 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 0x562b3b57ddc0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c0ad5c0 tx=0x7f3d9c018420 comp rx=0 tx=0).stop 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.524+0000 7f3dadc52740 1 -- 172.21.15.16:0/1346234802 shutdown_connections 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b593c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1ac1b0 0x562b3b589870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 --2- 172.21.15.16:0/1346234802 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1939f0 0x562b3b57ddc0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 -- 172.21.15.16:0/1346234802 >> 172.21.15.16:0/1346234802 conn(0x562b3b1c9460 msgr2=0x562b3b5860e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 -- 172.21.15.16:0/1346234802 shutdown_connections 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 -- 172.21.15.16:0/1346234802 wait complete. 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 -- start start 2024-09-21T03:00:56.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.525+0000 7f3dadc52740 1 Processor -- start 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 0x562b3b58bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b58c280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ac230 con 0x562b3b1b4390 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ad6a0 con 0x562b3b1ac1b0 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3da01ac900 con 0x562b3b1939f0 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3dadc52740 10 client.0 _ll_register_callbacks cb 0x7f3d9c0a9f80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3da5c7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3da5c7b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60466/0 (socket says 172.21.15.16:60466) 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3da5c7b640 1 -- 172.21.15.16:0/2155468461 learned_addr learned my addr 172.21.15.16:0/2155468461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3da547a640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 0x562b3b58bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.526+0000 7f3da4c79640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b58c280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 -- 172.21.15.16:0/2155468461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 msgr2=0x562b3b58bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 0x562b3b58bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 -- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 msgr2=0x562b3b58c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b58c280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 -- 172.21.15.16:0/2155468461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b3b603790 con 0x562b3b1b4390 2024-09-21T03:00:56.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da547a640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 0x562b3b58bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:56.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3da5c7b640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d9802f340 tx=0x7f3d9802a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4805aa7e37e312e5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3d88ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d98003b80 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.527+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f3d98003d20 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3d9c0ae010 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f3d98004030 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3dadc52740 10 client.15531 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562b3b604a10 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d9c0ae010 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f3d98054070 con 0x562b3b1b4390 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.528+0000 7f3dadc52740 10 client.15531 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3dadc52740 10 client.15531 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:56.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3dadc52740 20 client.15531 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1d5070 0x562b3b607230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3dadc52740 4 client.15531 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:00:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b3b4f08a0 con 0x562b3b1d5070 2024-09-21T03:00:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3da547a640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1d5070 0x562b3b607230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:56.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.529+0000 7f3d88ff9640 10 client.15531 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3da547a640 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1d5070 0x562b3b607230 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8638d7a3e2c954fb server_cookie=3018ebdea7e7be73 in_seq=0 out_seq=0 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3d88ff9640 1 -- 172.21.15.16:0/2155468461 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b3b4f08a0 con 0x562b3b1d5070 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3d88ff9640 10 client.15531 handle_command_reply: tid=0 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 client.15531 shutdown 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 msgr2=0x562b3b58c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f3d9802f340 tx=0x7f3d9802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 shutdown_connections 2024-09-21T03:00:56.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562b3b1d5070 0x562b3b607230 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562b3b1b4390 0x562b3b58c7c0 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562b3b1ac1b0 0x562b3b58c280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 --2- 172.21.15.16:0/2155468461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562b3b1939f0 0x562b3b58bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 >> 172.21.15.16:0/2155468461 conn(0x562b3b1c9460 msgr2=0x562b3b5895b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.530+0000 7f3d88ff9640 0 client.15531 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.531+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 shutdown_connections 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.531+0000 7f3dadc52740 1 -- 172.21.15.16:0/2155468461 wait complete. 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.531+0000 7f3dadc52740 20 client.15531 trim_cache size 0 max 16384 2024-09-21T03:00:56.534 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.531+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 msgr2=0x7f3d8c0b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 0x7f3d8c0b8b00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f3d9802f500 tx=0x7f3d98031bc0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 msgr2=0x7f3da019a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f3d9c00e9c0 tx=0x7f3d9c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 shutdown_connections 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f3d8c0b65f0 0x7f3d8c0b8b00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3da0106bd0 0x7f3da019fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3da0106280 0x7f3da019aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 --2- 172.21.15.16:0/1865215340 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f3da0102030 0x7f3da019a4d0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:56.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 >> 172.21.15.16:0/1865215340 conn(0x7f3da0077c60 msgr2=0x7f3da0107510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 shutdown_connections 2024-09-21T03:00:56.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:56.532+0000 7f3da76de640 1 -- 172.21.15.16:0/1865215340 wait complete. 2024-09-21T03:00:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:56 smithi018 ceph-mon[34587]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:56.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:56 smithi123 ceph-mon[34645]: pgmap v338: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.7 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:58.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:00:58.232 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:00:58.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:00:58 smithi016 ceph-mon[31253]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:58.618 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:00:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:00:58 smithi018 ceph-mon[34587]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:00:58 smithi123 ceph-mon[34645]: pgmap v339: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:00:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 -- 172.21.15.16:0/824445449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8108480 msgr2=0x7f39e810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 --2- 172.21.15.16:0/824445449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8108480 0x7f39e810ecb0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f39e0009be0 tx=0x7f39e002a6c0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 -- 172.21.15.16:0/824445449 shutdown_connections 2024-09-21T03:00:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 --2- 172.21.15.16:0/824445449 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e8107b30 0x7f39e8107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 --2- 172.21.15.16:0/824445449 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8108480 0x7f39e810ecb0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 --2- 172.21.15.16:0/824445449 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e81021f0 0x7f39e8102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 -- 172.21.15.16:0/824445449 >> 172.21.15.16:0/824445449 conn(0x7f39e8077c60 msgr2=0x7f39e80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 -- 172.21.15.16:0/824445449 shutdown_connections 2024-09-21T03:00:59.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.223+0000 7f39efa69640 1 -- 172.21.15.16:0/824445449 wait complete. 2024-09-21T03:00:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.224+0000 7f39efa69640 1 Processor -- start 2024-09-21T03:00:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.224+0000 7f39efa69640 1 -- start start 2024-09-21T03:00:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 0x7f39e81a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e8108480 0x7f39e81a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81118c0 con 0x7f39e81021f0 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8111740 con 0x7f39e8108480 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39efa69640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39e8111a40 con 0x7f39e8107b30 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed004640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed004640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44800/0 (socket says 172.21.15.16:44800) 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed004640 1 -- 172.21.15.16:0/742701266 learned_addr learned my addr 172.21.15.16:0/742701266 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ee006640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e8108480 0x7f39e81a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed805640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 0x7f39e81a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed004640 1 -- 172.21.15.16:0/742701266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e8108480 msgr2=0x7f39e81a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.225+0000 7f39ed004640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e8108480 0x7f39e81a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.226+0000 7f39ed004640 1 -- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 msgr2=0x7f39e81a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.226+0000 7f39ed004640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 0x7f39e81a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.226+0000 7f39ed004640 1 -- 172.21.15.16:0/742701266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39e81a69b0 con 0x7f39e8107b30 2024-09-21T03:00:59.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.226+0000 7f39ed805640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 0x7f39e81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.226+0000 7f39ed004640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d800ef60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.227+0000 7f39deffd640 1 -- 172.21.15.16:0/742701266 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d800cd70 con 0x7f39e8107b30 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.227+0000 7f39deffd640 1 -- 172.21.15.16:0/742701266 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39d80035f0 con 0x7f39e8107b30 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.227+0000 7f39deffd640 1 -- 172.21.15.16:0/742701266 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8016660 con 0x7f39e8107b30 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.227+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39e81a6ca0 con 0x7f39e8107b30 2024-09-21T03:00:59.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.227+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39e81ad400 con 0x7f39e8107b30 2024-09-21T03:00:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.230+0000 7f39f5fdd740 1 Processor -- start 2024-09-21T03:00:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 -- start start 2024-09-21T03:00:59.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98151970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da98151eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da980c7e60 con 0x55da97d89670 2024-09-21T03:00:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55da980b7290 con 0x55da97daa350 2024-09-21T03:00:59.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.231+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da980bb0a0 con 0x55da97d81490 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:46952/0 (socket says 172.21.15.16:46952) 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 -- 172.21.15.16:0/2411289471 learned_addr learned my addr 172.21.15.16:0/2411289471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ed805640 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98151970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 -- 172.21.15.16:0/2411289471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 msgr2=0x55da98151970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98151970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 -- 172.21.15.16:0/2411289471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 msgr2=0x55da98151eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:00:59.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da98151eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 -- 172.21.15.16:0/2411289471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da97e15930 con 0x55da97daa350 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.232+0000 7f39ee006640 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39e002acb0 tx=0x7f39e0030420 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5bed7808171270da server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.233+0000 7f39d3fff640 1 -- 172.21.15.16:0/2411289471 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0041030 con 0x55da97daa350 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.233+0000 7f39d3fff640 1 -- 172.21.15.16:0/2411289471 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e0003e20 con 0x55da97daa350 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.233+0000 7f39d3fff640 1 -- 172.21.15.16:0/2411289471 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0036040 con 0x55da97daa350 2024-09-21T03:00:59.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.233+0000 7f39deffd640 1 -- 172.21.15.16:0/742701266 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f39d801d020 con 0x7f39e8107b30 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.234+0000 7f39deffd640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 0x7f39d40b8b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.234+0000 7f39deffd640 1 -- 172.21.15.16:0/742701266 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f39d80e71b0 con 0x7f39e8107b30 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.234+0000 7f39ed805640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 0x7f39d40b8b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2411289471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 msgr2=0x55da98151430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f39e002acb0 tx=0x7f39e0030420 comp rx=0 tx=0).stop 2024-09-21T03:00:59.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2411289471 shutdown_connections 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da98151eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98151970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2411289471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98151430 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.235+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2411289471 >> 172.21.15.16:0/2411289471 conn(0x55da97d9e740 msgr2=0x55da9814f320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.236+0000 7f39ed805640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 0x7f39d40b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55da981500b0 tx=0x7f39e4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.236+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2411289471 shutdown_connections 2024-09-21T03:00:59.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.236+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2411289471 wait complete. 2024-09-21T03:00:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.237+0000 7f39f5fdd740 1 -- start start 2024-09-21T03:00:59.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.237+0000 7f39f5fdd740 1 Processor -- start 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.237+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.237+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d89670 0x55da9815d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.237+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98168ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55da97f18890 con 0x55da97d81490 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55da98051d40 con 0x55da97daa350 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39f5fdd740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55da9815fbb0 con 0x55da97d89670 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ed805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39f5fdd740 10 client.0 _ll_register_callbacks cb 0x55da980b5c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ed805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60488/0 (socket says 172.21.15.16:60488) 2024-09-21T03:00:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ed805640 1 -- 172.21.15.16:0/2588321930 learned_addr learned my addr 172.21.15.16:0/2588321930 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98168ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 -- 172.21.15.16:0/2588321930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d89670 msgr2=0x55da9815d530 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d89670 0x55da9815d530 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 -- 172.21.15.16:0/2588321930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 msgr2=0x55da9815cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ee006640 1 -- 172.21.15.16:0/2588321930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da98169490 con 0x55da97daa350 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.238+0000 7f39ed805640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39ee006640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98168ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034040 tx=0x7f39e0003c20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=941e97b3787b17dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e0036050 con 0x55da97daa350 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e0003e20 con 0x55da97daa350 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f39e0030b70 con 0x55da97daa350 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f39e0041040 con 0x55da97daa350 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39f5fdd740 10 client.25250 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:59.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.239+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55da9816bda0 con 0x55da97daa350 2024-09-21T03:00:59.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e0003e20 con 0x55da97daa350 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f39e003d4f0 con 0x55da97daa350 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 10 client.25250 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 10 client.25250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 20 client.25250 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d68cd0 0x55da9816f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 4 client.25250 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da980c9db0 con 0x55da97d68cd0 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.240+0000 7f39ed805640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d68cd0 0x55da9816f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.241+0000 7f39d17fa640 10 client.25250 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:59.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.241+0000 7f39ed805640 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d68cd0 0x55da9816f100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb5e2a43d8dbd10a server_cookie=54b64fecc79abf00 in_seq=0 out_seq=0 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39d17fa640 1 -- 172.21.15.16:0/2588321930 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55da980c9db0 con 0x55da97d68cd0 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39d17fa640 10 client.25250 handle_command_reply: tid=0 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39f5fdd740 1 client.25250 shutdown 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 msgr2=0x55da98168ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98168ea0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f39e0034040 tx=0x7f39e0003c20 comp rx=0 tx=0).stop 2024-09-21T03:00:59.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.242+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 shutdown_connections 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d68cd0 0x55da9816f100 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98168ea0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d89670 0x55da9815d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2588321930 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da9815cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39d17fa640 0 client.25250 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 >> 172.21.15.16:0/2588321930 conn(0x55da97d9e740 msgr2=0x55da98154610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 shutdown_connections 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2588321930 wait complete. 2024-09-21T03:00:59.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.243+0000 7f39f5fdd740 20 client.25250 trim_cache size 0 max 16384 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.279+0000 7f39f5fdd740 1 Processor -- start 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 -- start start 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 0x55da98168e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39ed805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39ed805640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60492/0 (socket says 172.21.15.16:60492) 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 0x55da98152e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39ed805640 1 -- 172.21.15.16:0/3329591138 learned_addr learned my addr 172.21.15.16:0/3329591138 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b3c70 con 0x55da97d81490 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b64f0 con 0x55da97d89670 2024-09-21T03:00:59.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.280+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b42e0 con 0x55da97daa350 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed004640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 0x55da98168e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 -- 172.21.15.16:0/3329591138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 msgr2=0x55da98152e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ee006640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 0x55da98152e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 0x55da98152e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 -- 172.21.15.16:0/3329591138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 msgr2=0x55da98168e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 0x55da98168e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 -- 172.21.15.16:0/3329591138 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da98153720 con 0x55da97d81490 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ee006640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 0x55da98152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed004640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 0x55da98168e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39ed805640 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009fb0 tx=0x7f39e4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5848fa285d1c6a5d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39d37fe640 1 -- 172.21.15.16:0/3329591138 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4021110 con 0x55da97d81490 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39d37fe640 1 -- 172.21.15.16:0/3329591138 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39e4021910 con 0x55da97d81490 2024-09-21T03:00:59.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.281+0000 7f39d37fe640 1 -- 172.21.15.16:0/3329591138 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39e4003850 con 0x55da97d81490 2024-09-21T03:00:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 msgr2=0x55da981585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f39e4009fb0 tx=0x7f39e4014000 comp rx=0 tx=0).stop 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 shutdown_connections 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97daa350 0x55da98152e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97d89670 0x55da98168e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/3329591138 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d81490 0x55da981585a0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 >> 172.21.15.16:0/3329591138 conn(0x55da97d9e740 msgr2=0x55da9815e310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 shutdown_connections 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- 172.21.15.16:0/3329591138 wait complete. 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 -- start start 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.283+0000 7f39f5fdd740 1 Processor -- start 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.284+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98170180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.284+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.284+0000 7f39f5fdd740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98170c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.284+0000 7f39ed004640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.284+0000 7f39ed004640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60506/0 (socket says 172.21.15.16:60506) 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed805640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98170180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed805640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98170180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44840/0 (socket says 172.21.15.16:44840) 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 -- 172.21.15.16:0/2876342563 learned_addr learned my addr 172.21.15.16:0/2876342563 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ee006640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98170c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b49b0 con 0x55da97d89670 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b5e20 con 0x55da97daa350 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f39e81b5080 con 0x55da97d81490 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 -- 172.21.15.16:0/2876342563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 msgr2=0x55da98170180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98170180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 -- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 msgr2=0x55da98170c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98170c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39ed004640 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da981d88d0 con 0x55da97d89670 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.285+0000 7f39f5fdd740 10 client.0 _ll_register_callbacks cb 0x7f39e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39ed004640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f39d800afd0 tx=0x7f39d8003a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4881f50375f2ee7c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39d17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39e4021910 con 0x55da97d89670 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39d17fa640 1 -- 172.21.15.16:0/2876342563 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39d8098370 con 0x55da97d89670 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39d17fa640 1 -- 172.21.15.16:0/2876342563 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f39d80ac340 con 0x55da97d89670 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39d17fa640 1 -- 172.21.15.16:0/2876342563 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f39d80ac580 con 0x55da97d89670 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39f5fdd740 10 client.15543 fetch_fsmap learned FSMap version 20 2024-09-21T03:00:59.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.286+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55da981d98d0 con 0x55da97d89670 2024-09-21T03:00:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39d17fa640 1 -- 172.21.15.16:0/2876342563 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f39d800b040 con 0x55da97d89670 2024-09-21T03:00:59.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 10 client.15543 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 10 client.15543 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 20 client.15543 populate_metadata read hostname 'smithi016' 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d3e080 0x55da981dc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 4 client.15543 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55da980c5860 con 0x55da97d3e080 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.287+0000 7f39ed805640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d3e080 0x55da981dc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.288+0000 7f39ed805640 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d3e080 0x55da981dc080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0efa7afc82bc8e server_cookie=9019a090c494fc45 in_seq=0 out_seq=0 2024-09-21T03:00:59.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.288+0000 7f39d17fa640 10 client.15543 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39d17fa640 1 -- 172.21.15.16:0/2876342563 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55da980c5860 con 0x55da97d3e080 2024-09-21T03:00:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39d17fa640 10 client.15543 handle_command_reply: tid=0 2024-09-21T03:00:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39f5fdd740 1 client.15543 shutdown 2024-09-21T03:00:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 msgr2=0x55da981706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7f39d800afd0 tx=0x7f39d8003a30 comp rx=0 tx=0).stop 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 shutdown_connections 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.290+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55da97d3e080 0x55da981dc080 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55da97d89670 0x55da981706c0 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39d17fa640 0 client.15543 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55da97daa350 0x55da98170c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 --2- 172.21.15.16:0/2876342563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55da97d81490 0x55da98170180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 >> 172.21.15.16:0/2876342563 conn(0x55da97d9e740 msgr2=0x55da98152bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 shutdown_connections 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 1 -- 172.21.15.16:0/2876342563 wait complete. 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.291+0000 7f39f5fdd740 20 client.15543 trim_cache size 0 max 16384 2024-09-21T03:00:59.293 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "bf89f1c3-0bf1-4a04-bd60-efd71bad1967", 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 msgr2=0x7f39d40b8b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 0x7f39d40b8b80 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x55da981500b0 tx=0x7f39e4005f90 comp rx=0 tx=0).stop 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 msgr2=0x7f39e81a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:00:59.294 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f39d80058c0 tx=0x7f39d800ef60 comp rx=0 tx=0).stop 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 shutdown_connections 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f39d40b6670 0x7f39d40b8b80 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f39e8108480 0x7f39e81a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f39e8107b30 0x7f39e81a0fe0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 --2- 172.21.15.16:0/742701266 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f39e81021f0 0x7f39e81a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.292+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 >> 172.21.15.16:0/742701266 conn(0x7f39e8077c60 msgr2=0x7f39e81038c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.293+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 shutdown_connections 2024-09-21T03:00:59.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:00:59.293+0000 7f39efa69640 1 -- 172.21.15.16:0/742701266 wait complete. 2024-09-21T03:01:00.028 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:01:00.379 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:00.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-mon[31253]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:00.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:01:00.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:01:00.406 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.679 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:01:00.403+0000 7f4f7eb12640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:01:00.679 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:01:00.405+0000 7f4f7eb12640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:01:00.679 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:01:00.405+0000 7f4f7eb12640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:01:00.679 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:01:00.405+0000 7f4f7eb12640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:01:00.679 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:01:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:01:00.405+0000 7f4f7eb12640 -1 client.0 error registering admin socket command: (17) File exists 2024-09-21T03:01:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:00 smithi018 ceph-mon[34587]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:00 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:01:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:00 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:00 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:01:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:00 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:00 smithi123 ceph-mon[34645]: pgmap v340: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:00 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:01:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:00 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:00 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:01:00.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:00 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 -- 172.21.15.16:0/1466710530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f8102730 msgr2=0x7f42f8110560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 --2- 172.21.15.16:0/1466710530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f8102730 0x7f42f8110560 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009ca0 tx=0x7f42f002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 -- 172.21.15.16:0/1466710530 shutdown_connections 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 --2- 172.21.15.16:0/1466710530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f8102730 0x7f42f8110560 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 --2- 172.21.15.16:0/1466710530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8101de0 0x7f42f81021f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 --2- 172.21.15.16:0/1466710530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42f81086f0 0x7f42f8108b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.926+0000 7f43000e6640 1 -- 172.21.15.16:0/1466710530 >> 172.21.15.16:0/1466710530 conn(0x7f42f8077c60 msgr2=0x7f42f80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.927+0000 7f43000e6640 1 -- 172.21.15.16:0/1466710530 shutdown_connections 2024-09-21T03:01:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.927+0000 7f43000e6640 1 -- 172.21.15.16:0/1466710530 wait complete. 2024-09-21T03:01:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.928+0000 7f43000e6640 1 Processor -- start 2024-09-21T03:01:00.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.928+0000 7f43000e6640 1 -- start start 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42f8101de0 0x7f42f819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f81086f0 0x7f42f819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8113aa0 con 0x7f42f81086f0 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8113920 con 0x7f42f8102730 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f43000e6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8113c20 con 0x7f42f8101de0 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f42fe683640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f81086f0 0x7f42f819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f42fd681640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f42fd681640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:47020/0 (socket says 172.21.15.16:47020) 2024-09-21T03:01:00.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.929+0000 7f42fd681640 1 -- 172.21.15.16:0/1769024386 learned_addr learned my addr 172.21.15.16:0/1769024386 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 -- 172.21.15.16:0/1769024386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42f8101de0 msgr2=0x7f42f819a4b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42f8101de0 0x7f42f819a4b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 -- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f81086f0 msgr2=0x7f42f819fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f81086f0 0x7f42f819fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 -- 172.21.15.16:0/1769024386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42f81a0420 con 0x7f42f8102730 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42fd681640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e9f0 tx=0x7f42e800eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42eeffd640 1 -- 172.21.15.16:0/1769024386 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800ccc0 con 0x7f42f8102730 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42f81a0710 con 0x7f42f8102730 2024-09-21T03:01:00.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42eeffd640 1 -- 172.21.15.16:0/1769024386 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42e80035f0 con 0x7f42f8102730 2024-09-21T03:01:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f42eeffd640 1 -- 172.21.15.16:0/1769024386 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42e800ce60 con 0x7f42f8102730 2024-09-21T03:01:00.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.930+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42f81a6e70 con 0x7f42f8102730 2024-09-21T03:01:00.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.933+0000 7f430665a740 1 Processor -- start 2024-09-21T03:01:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.933+0000 7f430665a740 1 -- start start 2024-09-21T03:01:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a30270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a30cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3029a6e50 con 0x55a302668390 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a302996280 con 0x55a3026601b0 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f430665a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a30299a090 con 0x55a302689070 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:47028/0 (socket says 172.21.15.16:47028) 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 -- 172.21.15.16:0/3804445951 learned_addr learned my addr 172.21.15.16:0/3804445951 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 -- 172.21.15.16:0/3804445951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 msgr2=0x55a302a30270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a30270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 -- 172.21.15.16:0/3804445951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 msgr2=0x55a302a30cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:01:00.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a30cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.934+0000 7f42fde82640 1 -- 172.21.15.16:0/3804445951 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a3026f4650 con 0x55a3026601b0 2024-09-21T03:01:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.935+0000 7f42fde82640 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009b50 tx=0x7f42f400e720 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3647f85304e0fbf8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.935+0000 7f42eeffd640 1 -- 172.21.15.16:0/1769024386 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 23) ==== 154586+0+0 (secure 0 0 0) 0x7f42e8010690 con 0x7f42f8102730 2024-09-21T03:01:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.935+0000 7f42e3fff640 1 -- 172.21.15.16:0/3804445951 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4009e40 con 0x55a3026601b0 2024-09-21T03:01:00.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.935+0000 7f42e3fff640 1 -- 172.21.15.16:0/3804445951 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f40054d0 con 0x55a3026601b0 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f42e3fff640 1 -- 172.21.15.16:0/3804445951 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4009e40 con 0x55a3026601b0 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f42eeffd640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 0x7f42e40b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f42eeffd640 1 -- 172.21.15.16:0/1769024386 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42e80e79f0 con 0x7f42f8102730 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f42fe683640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 0x7f42e40b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f430665a740 1 -- 172.21.15.16:0/3804445951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 msgr2=0x55a302a307b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.936+0000 7f430665a740 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f42f4009b50 tx=0x7f42f400e720 comp rx=0 tx=0).stop 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 -- 172.21.15.16:0/3804445951 shutdown_connections 2024-09-21T03:01:00.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a30cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a307b0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 --2- 172.21.15.16:0/3804445951 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a30270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 -- 172.21.15.16:0/3804445951 >> 172.21.15.16:0/3804445951 conn(0x55a30267d460 msgr2=0x55a302a2e160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f42fe683640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 0x7f42e40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006010 tx=0x7f42f0009460 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 -- 172.21.15.16:0/3804445951 shutdown_connections 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.937+0000 7f430665a740 1 -- 172.21.15.16:0/3804445951 wait complete. 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 -- start start 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 Processor -- start 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a3b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.942 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a3027f7700 con 0x55a302668390 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a302931110 con 0x55a3026601b0 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a302a42eb0 con 0x55a302689070 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.938+0000 7f430665a740 10 client.0 _ll_register_callbacks cb 0x55a30294a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fd681640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a3b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44890/0 (socket says 172.21.15.16:44890) 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 -- 172.21.15.16:0/1050676648 learned_addr learned my addr 172.21.15.16:0/1050676648 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fde82640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 -- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 msgr2=0x55a302a3df50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3df50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 -- 172.21.15.16:0/1050676648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 msgr2=0x55a302a3b7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a3b7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 -- 172.21.15.16:0/1050676648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a30295ec40 con 0x55a302689070 2024-09-21T03:01:00.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42fe683640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009650 tx=0x7f42f002a8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc6df6b05d0902b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.939+0000 7f42e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f0003e40 con 0x55a302689070 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42f0004040 con 0x55a302689070 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f40054d0 con 0x55a302689070 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f42f0004240 con 0x55a302689070 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f430665a740 10 client.25195 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a302a44810 con 0x55a302689070 2024-09-21T03:01:00.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.940+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f002f660 con 0x55a302689070 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f42f0067020 con 0x55a302689070 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 10 client.25195 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 10 client.25195 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 20 client.25195 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a3026479f0 0x55a302a47ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 4 client.25195 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f42f40054d0 con 0x55a3026479f0 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.941+0000 7f42fde82640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a3026479f0 0x55a302a47ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.942+0000 7f42e17fa640 10 client.25195 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.942+0000 7f42fde82640 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a3026479f0 0x55a302a47ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2320d32e738e7e46 server_cookie=f1a8d6b6c0b17feb in_seq=0 out_seq=0 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f42e17fa640 1 -- 172.21.15.16:0/1050676648 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f42f40054d0 con 0x55a3026479f0 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f42e17fa640 10 client.25195 handle_command_reply: tid=0 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 client.25195 shutdown 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 msgr2=0x55a302a3be20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42f0009650 tx=0x7f42f002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 shutdown_connections 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a3026479f0 0x55a302a47ce0 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f42e17fa640 0 client.25195 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a3b7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302689070 0x55a302a3be20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 --2- 172.21.15.16:0/1050676648 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 >> 172.21.15.16:0/1050676648 conn(0x55a30267d460 msgr2=0x55a302a33400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 shutdown_connections 2024-09-21T03:01:00.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 1 -- 172.21.15.16:0/1050676648 wait complete. 2024-09-21T03:01:00.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.943+0000 7f430665a740 20 client.25195 trim_cache size 0 max 16384 2024-09-21T03:01:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.978+0000 7f430665a740 1 Processor -- start 2024-09-21T03:01:00.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.978+0000 7f430665a740 1 -- start start 2024-09-21T03:01:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302668390 0x55a302a3d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8071330 con 0x55a3026479f0 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8073bb0 con 0x55a3026601b0 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f430665a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80719a0 con 0x55a302668390 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f42fde82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f42fde82640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:60562/0 (socket says 172.21.15.16:60562) 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f42fde82640 1 -- 172.21.15.16:0/2913729236 learned_addr learned my addr 172.21.15.16:0/2913729236 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:00.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f42fe683640 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302668390 0x55a302a3d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.979+0000 7f42fd681640 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 -- 172.21.15.16:0/2913729236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302668390 msgr2=0x55a302a3d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302668390 0x55a302a3d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 -- 172.21.15.16:0/2913729236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 msgr2=0x55a302a3d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 -- 172.21.15.16:0/2913729236 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a302a3cab0 con 0x55a3026479f0 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.980+0000 7f42fde82640 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f4009770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2497fdc65b0336e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.981+0000 7f42e37fe640 1 -- 172.21.15.16:0/2913729236 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f401d040 con 0x55a3026479f0 2024-09-21T03:01:00.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.981+0000 7f42e37fe640 1 -- 172.21.15.16:0/2913729236 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f4030070 con 0x55a3026479f0 2024-09-21T03:01:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.981+0000 7f42e37fe640 1 -- 172.21.15.16:0/2913729236 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f40053d0 con 0x55a3026479f0 2024-09-21T03:01:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 -- 172.21.15.16:0/2913729236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 msgr2=0x55a302a3cd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f42f4000c00 tx=0x7f42f4009770 comp rx=0 tx=0).stop 2024-09-21T03:01:00.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 -- 172.21.15.16:0/2913729236 shutdown_connections 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a302668390 0x55a302a3d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a3d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 --2- 172.21.15.16:0/2913729236 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a3026479f0 0x55a302a3cd10 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 -- 172.21.15.16:0/2913729236 >> 172.21.15.16:0/2913729236 conn(0x55a30267d460 msgr2=0x55a302a2da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 -- 172.21.15.16:0/2913729236 shutdown_connections 2024-09-21T03:01:00.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.982+0000 7f430665a740 1 -- 172.21.15.16:0/2913729236 wait complete. 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 -- start start 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 Processor -- start 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a41630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a4d680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8072070 con 0x55a302668390 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42f80734e0 con 0x55a3026601b0 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42f8072740 con 0x55a3026479f0 2024-09-21T03:01:00.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.983+0000 7f430665a740 10 client.0 _ll_register_callbacks cb 0x7f42f4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fe683640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a4d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44906/0 (socket says 172.21.15.16:44906) 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 -- 172.21.15.16:0/1815073517 learned_addr learned my addr 172.21.15.16:0/1815073517 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fd681640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a41630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 -- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 msgr2=0x55a302a41630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a41630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 -- 172.21.15.16:0/1815073517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 msgr2=0x55a302a4d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a4d680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 -- 172.21.15.16:0/1815073517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a302a4de10 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fe683640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a4d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42fde82640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42f40058c0 tx=0x7f42f4009ee0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2ee4f911b9a4a6f5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.984+0000 7f42e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42f4039060 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42f4004da0 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42f4030070 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f42f4004680 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f430665a740 10 client.25201 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a302a51100 con 0x55a3026479f0 2024-09-21T03:01:00.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42f4004820 con 0x55a3026479f0 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f42f4034870 con 0x55a3026479f0 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f430665a740 10 client.25201 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.985+0000 7f430665a740 10 client.25201 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.986+0000 7f430665a740 20 client.25201 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.986+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a302689070 0x55a302acbe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.986+0000 7f430665a740 4 client.25201 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:01:00.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.986+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f42f4030070 con 0x55a302689070 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.986+0000 7f42fd681640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a302689070 0x55a302acbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.987+0000 7f42fd681640 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a302689070 0x55a302acbe30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d31801aacdce6573 server_cookie=efbd92874ecb79b8 in_seq=0 out_seq=0 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.987+0000 7f42e17fa640 10 client.25201 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f42e17fa640 1 -- 172.21.15.16:0/1815073517 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1138 (crc 0 0 0) 0x7f42f4030070 con 0x55a302689070 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f42e17fa640 10 client.25201 handle_command_reply: tid=0 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 client.25201 shutdown 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 msgr2=0x55a302a41070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f42f40058c0 tx=0x7f42f4009ee0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 shutdown_connections 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a302689070 0x55a302acbe30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f42e17fa640 0 client.25201 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a302668390 0x55a302a4d680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a3026601b0 0x55a302a41630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 --2- 172.21.15.16:0/1815073517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a3026479f0 0x55a302a41070 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 >> 172.21.15.16:0/1815073517 conn(0x55a30267d460 msgr2=0x55a302a3cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 shutdown_connections 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 1 -- 172.21.15.16:0/1815073517 wait complete. 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.988+0000 7f430665a740 20 client.25201 trim_cache size 0 max 16384 2024-09-21T03:01:00.992 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "bf89f1c3-0bf1-4a04-bd60-efd71bad1967", 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:00.993 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:00.994 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:00.995 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:00.995 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.989+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 msgr2=0x7f42e40b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.989+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 0x7f42e40b8b00 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f42f0006010 tx=0x7f42f0009460 comp rx=0 tx=0).stop 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.989+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 msgr2=0x7f42f819aa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.989+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f42e800e9f0 tx=0x7f42e800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.989+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 shutdown_connections 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42e40b65f0 0x7f42e40b8b00 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42f81086f0 0x7f42f819fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42f8102730 0x7f42f819aa50 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 --2- 172.21.15.16:0/1769024386 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42f8101de0 0x7f42f819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 >> 172.21.15.16:0/1769024386 conn(0x7f42f8077c60 msgr2=0x7f42f81063e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 shutdown_connections 2024-09-21T03:01:00.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:00.990+0000 7f43000e6640 1 -- 172.21.15.16:0/1769024386 wait complete. 2024-09-21T03:01:01.728 INFO:tasks.cephfs.filesystem:scrub status for tag:bf89f1c3-0bf1-4a04-bd60-efd71bad1967 - {'path': '/', 'tag': 'bf89f1c3-0bf1-4a04-bd60-efd71bad1967', 'options': 'recursive,force'} 2024-09-21T03:01:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:02 smithi018 ceph-mon[34587]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:02 smithi018 ceph-mon[34587]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:01:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:02 smithi123 ceph-mon[34645]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:02 smithi123 ceph-mon[34645]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:01:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:02 smithi016 ceph-mon[31253]: pgmap v341: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s rd, 682 B/s wr, 2 op/s 2024-09-21T03:01:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:02 smithi016 ceph-mon[31253]: mgrmap e24: x(active, since 9m), standbys: z, y 2024-09-21T03:01:03.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:03 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:01:03.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:03 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:01:03.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:03 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:01:04.583 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:04 smithi018 ceph-mon[34587]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:01:04.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:04 smithi123 ceph-mon[34645]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:01:04.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:04 smithi016 ceph-mon[31253]: pgmap v342: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 1 op/s 2024-09-21T03:01:06.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:06 smithi018 ceph-mon[34587]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:06.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:06 smithi123 ceph-mon[34645]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:06.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:06 smithi016 ceph-mon[31253]: pgmap v343: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:08 smithi018 ceph-mon[34587]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:08 smithi123 ceph-mon[34645]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:08.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:08 smithi016 ceph-mon[31253]: pgmap v344: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:10.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:10 smithi018 ceph-mon[34587]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:10.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:10 smithi123 ceph-mon[34645]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:10.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:10 smithi016 ceph-mon[31253]: pgmap v345: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:01:12.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:12 smithi018 ceph-mon[34587]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:12.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:12 smithi123 ceph-mon[34645]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:12.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:12 smithi016 ceph-mon[31253]: pgmap v346: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:14 smithi018 ceph-mon[34587]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:14 smithi123 ceph-mon[34645]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:14.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:14 smithi016 ceph-mon[31253]: pgmap v347: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:16 smithi018 ceph-mon[34587]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:16.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:16 smithi123 ceph-mon[34645]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:16.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:16 smithi016 ceph-mon[31253]: pgmap v348: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 17 op/s 2024-09-21T03:01:18.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:18 smithi018 ceph-mon[34587]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:18.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:18 smithi123 ceph-mon[34645]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:18.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:18 smithi016 ceph-mon[31253]: pgmap v349: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:20.620 INFO:tasks.workunit.client.0.smithi016.stderr:Updating files: 91% (11570/12697) Updating files: 92% (11682/12697) Updating files: 93% (11809/12697) Updating files: 94% (11936/12697) Updating files: 95% (12063/12697) Updating files: 96% (12190/12697) Updating files: 97% (12317/12697) Updating files: 98% (12444/12697) Updating files: 99% (12571/12697) Updating files: 100% (12697/12697) Updating files: 100% (12697/12697), done. 2024-09-21T03:01:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:20 smithi018 ceph-mon[34587]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:20.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:20 smithi123 ceph-mon[34645]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:20.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:20 smithi016 ceph-mon[31253]: pgmap v350: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:01:21.427 INFO:tasks.workunit.client.0.smithi016.stderr:Note: switching to '1843221895adccfb3b692060251dd6a0183bc410'. 2024-09-21T03:01:21.427 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.427 INFO:tasks.workunit.client.0.smithi016.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-09-21T03:01:21.427 INFO:tasks.workunit.client.0.smithi016.stderr:changes and commit them, and you can discard any commits you make in this 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:state without impacting any branches by switching back to a branch. 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:If you want to create a new branch to retain commits you create, you may 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:do so (now or later) by using -c with the switch command. Example: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: git switch -c 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:Or undo this operation with: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: git switch - 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:21.428 INFO:tasks.workunit.client.0.smithi016.stderr:HEAD is now at 1843221895a Merge PR #56776 into wip-pdonnell-testing-20240920.212106-debug 2024-09-21T03:01:21.433 DEBUG:teuthology.orchestra.run.smithi016:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-09-21T03:01:21.459 INFO:tasks.workunit.client.0.smithi016.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-09-21T03:01:21.462 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:01:21.462 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-09-21T03:01:22.179 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-09-21T03:01:22.254 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-09-21T03:01:22.296 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-09-21T03:01:22.298 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:01:22.298 INFO:tasks.workunit.client.0.smithi016.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-09-21T03:01:22.339 INFO:tasks.workunit.client.0.smithi016.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-09-21T03:01:22.342 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:01:22.342 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-09-21T03:01:22.399 INFO:tasks.workunit:Running workunits matching suites/iogen.sh on client.0... 2024-09-21T03:01:22.400 INFO:tasks.workunit:Running workunit suites/iogen.sh... 2024-09-21T03:01:22.400 DEBUG:teuthology.orchestra.run.smithi016:workunit test suites/iogen.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=1843221895adccfb3b692060251dd6a0183bc410 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/suites/iogen.sh 2024-09-21T03:01:22.462 INFO:tasks.workunit.client.0.smithi016.stdout:getting iogen 2024-09-21T03:01:22.462 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'getting iogen' 2024-09-21T03:01:22.462 INFO:tasks.workunit.client.0.smithi016.stderr:+ wget http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T03:01:22.482 INFO:tasks.workunit.client.0.smithi016.stderr:--2024-09-21 03:01:22-- http://download.ceph.com/qa/iogen_3.1p0.tar 2024-09-21T03:01:22.491 INFO:tasks.workunit.client.0.smithi016.stderr:Resolving download.ceph.com (download.ceph.com)... 158.69.68.124, 2607:5300:201:2000::3:58a1 2024-09-21T03:01:22.513 INFO:tasks.workunit.client.0.smithi016.stderr:Connecting to download.ceph.com (download.ceph.com)|158.69.68.124|:80... connected. 2024-09-21T03:01:22.534 INFO:tasks.workunit.client.0.smithi016.stderr:HTTP request sent, awaiting response... 200 OK 2024-09-21T03:01:22.534 INFO:tasks.workunit.client.0.smithi016.stderr:Length: 8071 (7.9K) [application/octet-stream] 2024-09-21T03:01:22.535 INFO:tasks.workunit.client.0.smithi016.stderr:Saving to: ‘iogen_3.1p0.tar’ 2024-09-21T03:01:22.538 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:22.538 INFO:tasks.workunit.client.0.smithi016.stderr: 0K ....... 100% 1.26G=0s 2024-09-21T03:01:22.539 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:22.539 INFO:tasks.workunit.client.0.smithi016.stderr:2024-09-21 03:01:22 (1.26 GB/s) - ‘iogen_3.1p0.tar’ saved [8071/8071] 2024-09-21T03:01:22.539 INFO:tasks.workunit.client.0.smithi016.stderr: 2024-09-21T03:01:22.539 INFO:tasks.workunit.client.0.smithi016.stderr:+ tar -xvzf iogen_3.1p0.tar 2024-09-21T03:01:22.557 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0 2024-09-21T03:01:22.558 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src 2024-09-21T03:01:22.558 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.8 2024-09-21T03:01:22.560 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/src/iogen.c 2024-09-21T03:01:22.561 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/README 2024-09-21T03:01:22.561 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile.bsd 2024-09-21T03:01:22.563 INFO:tasks.workunit.client.0.smithi016.stdout:iogen_3.1p0/Makefile 2024-09-21T03:01:22.564 INFO:tasks.workunit.client.0.smithi016.stderr:+ cd iogen_3.1p0 2024-09-21T03:01:22.564 INFO:tasks.workunit.client.0.smithi016.stdout:making iogen 2024-09-21T03:01:22.564 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'making iogen' 2024-09-21T03:01:22.565 INFO:tasks.workunit.client.0.smithi016.stderr:+ make 2024-09-21T03:01:22.576 INFO:tasks.workunit.client.0.smithi016.stdout:gcc -D_FILE_OFFSET_BITS=64 -DVERSION=\"3.1p0\" -Dstrlcpy=strncpy -Darc4random=rand -O2 src/iogen.c -o iogen 2024-09-21T03:01:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:22 smithi018 ceph-mon[34587]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:01:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:22 smithi123 ceph-mon[34645]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:01:22.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:22 smithi016 ceph-mon[31253]: pgmap v351: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:01:22.799 INFO:tasks.workunit.client.0.smithi016.stdout:nroff -mandoc -Tascii src/iogen.8 > iogen.cat8 2024-09-21T03:01:22.928 INFO:tasks.workunit.client.0.smithi016.stdout:running iogen 2024-09-21T03:01:22.928 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'running iogen' 2024-09-21T03:01:22.928 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -n 5 -s 2g 2024-09-21T03:01:22.929 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 83624 2024-09-21T03:01:22.929 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 83625 2024-09-21T03:01:22.929 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 83626 2024-09-21T03:01:22.929 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 83627 2024-09-21T03:01:22.929 INFO:tasks.workunit.client.0.smithi016.stderr:child spawned 83628 2024-09-21T03:01:22.930 INFO:tasks.workunit.client.0.smithi016.stdout:sleep for 10 min 2024-09-21T03:01:22.930 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'sleep for 10 min' 2024-09-21T03:01:22.930 INFO:tasks.workunit.client.0.smithi016.stderr:+ sleep 600 2024-09-21T03:01:24.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:24 smithi018 ceph-mon[34587]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:01:24.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:24 smithi123 ceph-mon[34645]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:01:24.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:24 smithi016 ceph-mon[31253]: pgmap v352: 97 pgs: 97 active+clean; 1.4 MiB data, 384 MiB used, 1.0 TiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:01:26.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:26 smithi018 ceph-mon[34587]: pgmap v353: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 53 MiB/s wr, 53 op/s 2024-09-21T03:01:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:26 smithi123 ceph-mon[34645]: pgmap v353: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 53 MiB/s wr, 53 op/s 2024-09-21T03:01:26.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:26 smithi016 ceph-mon[31253]: pgmap v353: 97 pgs: 97 active+clean; 639 MiB data, 1.4 GiB used, 1.0 TiB / 1.0 TiB avail; 121 KiB/s rd, 53 MiB/s wr, 53 op/s 2024-09-21T03:01:28.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:28 smithi018 ceph-mon[34587]: pgmap v354: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 86 MiB/s wr, 61 op/s 2024-09-21T03:01:28.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:28 smithi123 ceph-mon[34645]: pgmap v354: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 86 MiB/s wr, 61 op/s 2024-09-21T03:01:28.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:28 smithi016 ceph-mon[31253]: pgmap v354: 97 pgs: 97 active+clean; 1.0 GiB data, 2.4 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 86 MiB/s wr, 61 op/s 2024-09-21T03:01:30.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:30 smithi018 ceph-mon[34587]: pgmap v355: 97 pgs: 97 active+clean; 2.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 204 MiB/s wr, 91 op/s 2024-09-21T03:01:30.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:30 smithi123 ceph-mon[34645]: pgmap v355: 97 pgs: 97 active+clean; 2.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 204 MiB/s wr, 91 op/s 2024-09-21T03:01:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:30 smithi016 ceph-mon[31253]: pgmap v355: 97 pgs: 97 active+clean; 2.4 GiB data, 5.7 GiB used, 1.0 TiB / 1.0 TiB avail; 120 KiB/s rd, 204 MiB/s wr, 91 op/s 2024-09-21T03:01:31.728 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:01:32.186 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:32.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:32 smithi016 ceph-mon[31253]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 398 MiB/s wr, 303 op/s 2024-09-21T03:01:32.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:32 smithi018 ceph-mon[34587]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 398 MiB/s wr, 303 op/s 2024-09-21T03:01:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:32 smithi123 ceph-mon[34645]: pgmap v356: 97 pgs: 97 active+clean; 4.7 GiB data, 13 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 398 MiB/s wr, 303 op/s 2024-09-21T03:01:33.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 -- 172.21.15.16:0/1468609243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a81094d0 msgr2=0x7f55a81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 --2- 172.21.15.16:0/1468609243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a81094d0 0x7f55a81098e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f559c009900 tx=0x7f559c02a3f0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 -- 172.21.15.16:0/1468609243 shutdown_connections 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 --2- 172.21.15.16:0/1468609243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a8109e20 0x7f55a8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 --2- 172.21.15.16:0/1468609243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a81094d0 0x7f55a81098e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 --2- 172.21.15.16:0/1468609243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a8103b90 0x7f55a8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 -- 172.21.15.16:0/1468609243 >> 172.21.15.16:0/1468609243 conn(0x7f55a80ff5e0 msgr2=0x7f55a8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 -- 172.21.15.16:0/1468609243 shutdown_connections 2024-09-21T03:01:33.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.350+0000 7f55ae495640 1 -- 172.21.15.16:0/1468609243 wait complete. 2024-09-21T03:01:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.355+0000 7f55ae495640 1 Processor -- start 2024-09-21T03:01:33.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.356+0000 7f55ae495640 1 -- start start 2024-09-21T03:01:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.357+0000 7f55ae495640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a8103b90 0x7f55a81980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.358+0000 7f55ae495640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.358+0000 7f55ae495640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109e20 0x7f55a819d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.359+0000 7f55ae495640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8111920 con 0x7f55a8103b90 2024-09-21T03:01:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.359+0000 7f55ae495640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81117a0 con 0x7f55a8109e20 2024-09-21T03:01:33.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.359+0000 7f55ae495640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55a8111aa0 con 0x7f55a81094d0 2024-09-21T03:01:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.360+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.360+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46162/0 (socket says 172.21.15.16:46162) 2024-09-21T03:01:33.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.360+0000 7f55a77fe640 1 -- 172.21.15.16:0/1507505422 learned_addr learned my addr 172.21.15.16:0/1507505422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 -- 172.21.15.16:0/1507505422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109e20 msgr2=0x7f55a819d910 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109e20 0x7f55a819d910 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 -- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a8103b90 msgr2=0x7f55a81980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a8103b90 0x7f55a81980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 -- 172.21.15.16:0/1507505422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f55a819e040 con 0x7f55a81094d0 2024-09-21T03:01:33.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.361+0000 7f55a77fe640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f559c02a910 tx=0x7f559c030650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.366 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.365+0000 7f55a57fa640 1 -- 172.21.15.16:0/1507505422 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c03f070 con 0x7f55a81094d0 2024-09-21T03:01:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.365+0000 7f55a57fa640 1 -- 172.21.15.16:0/1507505422 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c004460 con 0x7f55a81094d0 2024-09-21T03:01:33.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.365+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55a819e2d0 con 0x7f55a81094d0 2024-09-21T03:01:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.367+0000 7f55a57fa640 1 -- 172.21.15.16:0/1507505422 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c035030 con 0x7f55a81094d0 2024-09-21T03:01:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.368+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55a819e6f0 con 0x7f55a81094d0 2024-09-21T03:01:33.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.372+0000 7f55a57fa640 1 -- 172.21.15.16:0/1507505422 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f559c042070 con 0x7f55a81094d0 2024-09-21T03:01:33.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.382+0000 7f55b4a09740 1 Processor -- start 2024-09-21T03:01:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.383+0000 7f55b4a09740 1 -- start start 2024-09-21T03:01:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f563070 0x560a4f90a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f90a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a4f880e50 con 0x560a4f563070 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560a4f870280 con 0x560a4f53a1b0 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.384+0000 7f55b4a09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a4f874090 con 0x560a4f542390 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.385+0000 7f55a57fa640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 0x7f55900b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.385+0000 7f55aca32640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 0x7f55900b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.386+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f90a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.386+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.386+0000 7f55a77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46170/0 (socket says 172.21.15.16:46170) 2024-09-21T03:01:33.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.386+0000 7f55a77fe640 1 -- 172.21.15.16:0/277984573 learned_addr learned my addr 172.21.15.16:0/277984573 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 -- 172.21.15.16:0/277984573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 msgr2=0x560a4f90a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f90a7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 -- 172.21.15.16:0/277984573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f563070 msgr2=0x560a4f90a270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f563070 0x560a4f90a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 -- 172.21.15.16:0/277984573 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4f5ce650 con 0x560a4f542390 2024-09-21T03:01:33.390 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.387+0000 7f55a77fe640 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f559c105320 tx=0x7f559c0cd350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=789ec9f69f418a5f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.392+0000 7f55a57fa640 1 -- 172.21.15.16:0/1507505422 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f559c107ca0 con 0x7f55a81094d0 2024-09-21T03:01:33.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.393+0000 7f558e7fc640 1 -- 172.21.15.16:0/277984573 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f559c0d2590 con 0x560a4f542390 2024-09-21T03:01:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.393+0000 7f558e7fc640 1 -- 172.21.15.16:0/277984573 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c0f5070 con 0x560a4f542390 2024-09-21T03:01:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.395+0000 7f55b4a09740 1 -- 172.21.15.16:0/277984573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 msgr2=0x560a4f90acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.395+0000 7f55b4a09740 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f559c105320 tx=0x7f559c0cd350 comp rx=0 tx=0).stop 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.396+0000 7f55aca32640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 0x7f55900b8c40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009810 tx=0x7f55a0009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.397+0000 7f55b4a09740 1 -- 172.21.15.16:0/277984573 shutdown_connections 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.397+0000 7f55b4a09740 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f90acf0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.397+0000 7f55b4a09740 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f90a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.397+0000 7f55b4a09740 1 --2- 172.21.15.16:0/277984573 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f563070 0x560a4f90a270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.397+0000 7f55b4a09740 1 -- 172.21.15.16:0/277984573 >> 172.21.15.16:0/277984573 conn(0x560a4f557460 msgr2=0x560a4f908160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.416+0000 7f55b4a09740 1 -- 172.21.15.16:0/277984573 shutdown_connections 2024-09-21T03:01:33.420 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.417+0000 7f55b4a09740 1 -- 172.21.15.16:0/277984573 wait complete. 2024-09-21T03:01:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.418+0000 7f55b4a09740 1 -- start start 2024-09-21T03:01:33.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.419+0000 7f55b4a09740 1 Processor -- start 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f9158a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f542390 0x560a4f915de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560a4f6d1700 con 0x560a4f542390 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560a4f80b110 con 0x560a4f53a1b0 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.420+0000 7f55b4a09740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560a4f921020 con 0x560a4f563070 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.421+0000 7f55b4a09740 10 client.0 _ll_register_callbacks cb 0x560a4f824990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.421+0000 7f55aca32640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.421+0000 7f55aca32640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46182/0 (socket says 172.21.15.16:46182) 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.421+0000 7f55aca32640 1 -- 172.21.15.16:0/720916898 learned_addr learned my addr 172.21.15.16:0/720916898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:33.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.422+0000 7f55a7fff640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f9158a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55a77fe640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f542390 0x560a4f915de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55aca32640 1 -- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 msgr2=0x560a4f9158a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55aca32640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f9158a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55aca32640 1 -- 172.21.15.16:0/720916898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f542390 msgr2=0x560a4f915de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55aca32640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f542390 0x560a4f915de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.426+0000 7f55aca32640 1 -- 172.21.15.16:0/720916898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4f838c40 con 0x560a4f563070 2024-09-21T03:01:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.427+0000 7f55aca32640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55a0003d10 tx=0x7f55a0004a40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4ab66132ec7684fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.427+0000 7f55877fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:01:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.427+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f55a0004cc0 con 0x560a4f563070 2024-09-21T03:01:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.428+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f559c0f5070 con 0x560a4f563070 2024-09-21T03:01:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.428+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f55a0009040 con 0x560a4f563070 2024-09-21T03:01:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.428+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f55a002c050 con 0x560a4f563070 2024-09-21T03:01:33.430 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.428+0000 7f55b4a09740 10 client.25213 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:33.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.428+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a4f92cc90 con 0x560a4f563070 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f559c0f5070 con 0x560a4f563070 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f55a000f040 con 0x560a4f563070 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 10 client.25213 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 10 client.25213 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 20 client.25213 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f5219f0 0x560a4f930680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 4 client.25213 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.430+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a4f882da0 con 0x560a4f5219f0 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.431+0000 7f55a7fff640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f5219f0 0x560a4f930680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.433+0000 7f55877fe640 10 client.25213 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:33.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.433+0000 7f55a7fff640 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f5219f0 0x560a4f930680 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=94d73151a4735a46 server_cookie=6e192831b2d8abfc in_seq=0 out_seq=0 2024-09-21T03:01:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.455+0000 7f55877fe640 1 -- 172.21.15.16:0/720916898 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560a4f882da0 con 0x560a4f5219f0 2024-09-21T03:01:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.455+0000 7f55877fe640 10 client.25213 handle_command_reply: tid=0 2024-09-21T03:01:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.455+0000 7f55b4a09740 1 client.25213 shutdown 2024-09-21T03:01:33.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.455+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 msgr2=0x560a4f916360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.455+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55a0003d10 tx=0x7f55a0004a40 comp rx=0 tx=0).stop 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 shutdown_connections 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f5219f0 0x560a4f930680 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f542390 0x560a4f915de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f916360 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 --2- 172.21.15.16:0/720916898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f53a1b0 0x560a4f9158a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 >> 172.21.15.16:0/720916898 conn(0x560a4f557460 msgr2=0x560a4f90d450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55877fe640 0 client.25213 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 shutdown_connections 2024-09-21T03:01:33.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.456+0000 7f55b4a09740 1 -- 172.21.15.16:0/720916898 wait complete. 2024-09-21T03:01:33.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.458+0000 7f55b4a09740 20 client.25213 trim_cache size 0 max 16384 2024-09-21T03:01:33.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.508+0000 7f55b4a09740 1 Processor -- start 2024-09-21T03:01:33.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.512+0000 7f55b4a09740 1 -- start start 2024-09-21T03:01:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.513+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.514+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.514+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41478/0 (socket says 172.21.15.16:41478) 2024-09-21T03:01:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.515+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f9174d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.515+0000 7f55a7fff640 1 -- 172.21.15.16:0/2331471692 learned_addr learned my addr 172.21.15.16:0/2331471692 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.516+0000 7f55b4a09740 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f563070 0x560a4f907710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.516+0000 7f55a77fe640 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f9174d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.517+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81ab350 con 0x560a4f53a1b0 2024-09-21T03:01:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.517+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81adbd0 con 0x560a4f563070 2024-09-21T03:01:33.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.517+0000 7f55aca32640 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f563070 0x560a4f907710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.518+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81ab9c0 con 0x560a4f542390 2024-09-21T03:01:33.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.519+0000 7f55a7fff640 1 -- 172.21.15.16:0/2331471692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 msgr2=0x560a4f9174d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.519+0000 7f55a7fff640 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f9174d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.519+0000 7f55a7fff640 1 -- 172.21.15.16:0/2331471692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f563070 msgr2=0x560a4f907710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.519+0000 7f55a7fff640 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f563070 0x560a4f907710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.519+0000 7f55a7fff640 1 -- 172.21.15.16:0/2331471692 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a4f908a50 con 0x560a4f53a1b0 2024-09-21T03:01:33.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.522+0000 7f55a7fff640 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x560a4f908ef0 tx=0x7f5598005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d252d912310a5d6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.524+0000 7f558dffb640 1 -- 172.21.15.16:0/2331471692 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598013da0 con 0x560a4f53a1b0 2024-09-21T03:01:33.527 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.524+0000 7f558dffb640 1 -- 172.21.15.16:0/2331471692 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5598005070 con 0x560a4f53a1b0 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 msgr2=0x560a4f90bc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x560a4f908ef0 tx=0x7f5598005850 comp rx=0 tx=0).stop 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 shutdown_connections 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f563070 0x560a4f907710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f542390 0x560a4f9174d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 --2- 172.21.15.16:0/2331471692 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f53a1b0 0x560a4f90bc60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.526+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 >> 172.21.15.16:0/2331471692 conn(0x560a4f557460 msgr2=0x560a4f916dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.528+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 shutdown_connections 2024-09-21T03:01:33.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.528+0000 7f55b4a09740 1 -- 172.21.15.16:0/2331471692 wait complete. 2024-09-21T03:01:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.534+0000 7f55b4a09740 1 -- start start 2024-09-21T03:01:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.534+0000 7f55b4a09740 1 Processor -- start 2024-09-21T03:01:33.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.535+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.536+0000 7f55b4a09740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f9219f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.536+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.536+0000 7f55a7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53644/0 (socket says 172.21.15.16:53644) 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.536+0000 7f55a7fff640 1 -- 172.21.15.16:0/3196802878 learned_addr learned my addr 172.21.15.16:0/3196802878 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.537+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f4f6da0 0x560a4f921f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.537+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81ac090 con 0x560a4f4f6da0 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.537+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81ad500 con 0x560a4f542390 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.537+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f55a81ac760 con 0x560a4f563070 2024-09-21T03:01:33.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.537+0000 7f55b4a09740 10 client.0 _ll_register_callbacks cb 0x7f55a0009c40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.540+0000 7f55aca32640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f4f6da0 0x560a4f921f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.539+0000 7f55a77fe640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f9219f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.541+0000 7f55a7fff640 1 -- 172.21.15.16:0/3196802878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 msgr2=0x560a4f9219f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.541+0000 7f55a7fff640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f9219f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.541+0000 7f55a7fff640 1 -- 172.21.15.16:0/3196802878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f4f6da0 msgr2=0x560a4f921f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.541+0000 7f55a7fff640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f4f6da0 0x560a4f921f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.541+0000 7f55a7fff640 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a4f922470 con 0x560a4f542390 2024-09-21T03:01:33.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.542+0000 7f55a7fff640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55980058c0 tx=0x7f5598004830 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a62e92d05e9997c6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:33.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.542+0000 7f55877fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:01:33.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.545+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5598013db0 con 0x560a4f542390 2024-09-21T03:01:33.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.545+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a4f87e850 con 0x560a4f542390 2024-09-21T03:01:33.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.546+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5598004030 con 0x560a4f542390 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.564+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5598004230 con 0x560a4f542390 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.564+0000 7f55b4a09740 10 client.25289 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.565+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560a4f99ad60 con 0x560a4f542390 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.565+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a4f87e850 con 0x560a4f542390 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.566+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5598036070 con 0x560a4f542390 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 10 client.25289 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 10 client.25289 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 20 client.25289 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f99b3b0 0x560a4f99d7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 4 client.25289 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:01:33.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.567+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560a4f87bc80 con 0x560a4f99b3b0 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.568+0000 7f55a77fe640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f99b3b0 0x560a4f99d7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.570+0000 7f55a77fe640 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f99b3b0 0x560a4f99d7d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5368047ad65d98b5 server_cookie=ab2dedf1fc0d616 in_seq=0 out_seq=0 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.570+0000 7f55877fe640 10 client.25289 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.570+0000 7f55877fe640 1 -- 172.21.15.16:0/3196802878 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+942 (crc 0 0 0) 0x560a4f87bc80 con 0x560a4f99b3b0 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.570+0000 7f55877fe640 10 client.25289 handle_command_reply: tid=0 2024-09-21T03:01:33.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.571+0000 7f55b4a09740 1 client.25289 shutdown 2024-09-21T03:01:33.593 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:01:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:33.596 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:01:33.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 msgr2=0x560a4f926540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f55980058c0 tx=0x7f5598004830 comp rx=0 tx=0).stop 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 shutdown_connections 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560a4f99b3b0 0x560a4f99d7d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560a4f4f6da0 0x560a4f921f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560a4f563070 0x560a4f9219f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 --2- 172.21.15.16:0/3196802878 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560a4f542390 0x560a4f926540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.573+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 >> 172.21.15.16:0/3196802878 conn(0x560a4f557460 msgr2=0x560a4f90ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.574+0000 7f55877fe640 0 client.25289 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.574+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 shutdown_connections 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.574+0000 7f55b4a09740 1 -- 172.21.15.16:0/3196802878 wait complete. 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.574+0000 7f55b4a09740 20 client.25289 trim_cache size 0 max 16384 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.579+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 msgr2=0x7f55900b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.579+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 0x7f55900b8c40 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f55a0009810 tx=0x7f55a0009760 comp rx=0 tx=0).stop 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.579+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 msgr2=0x7f55a8198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.579+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f559c02a910 tx=0x7f559c030650 comp rx=0 tx=0).stop 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 shutdown_connections 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f55900b6730 0x7f55900b8c40 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f55a8109e20 0x7f55a819d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f55a81094d0 0x7f55a8198670 secure :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f559c02a910 tx=0x7f559c030650 comp rx=0 tx=0).stop 2024-09-21T03:01:33.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 --2- 172.21.15.16:0/1507505422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f55a8103b90 0x7f55a81980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 >> 172.21.15.16:0/1507505422 conn(0x7f55a80ff5e0 msgr2=0x7f55a8106960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 shutdown_connections 2024-09-21T03:01:33.599 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:33.580+0000 7f55ae495640 1 -- 172.21.15.16:0/1507505422 wait complete. 2024-09-21T03:01:33.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:33 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:33.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:33 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:34.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:33 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:01:35.063 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:01:35.063 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:01:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: pgmap v357: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 437 MiB/s wr, 357 op/s 2024-09-21T03:01:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:34 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: pgmap v357: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 437 MiB/s wr, 357 op/s 2024-09-21T03:01:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:34 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.285 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: pgmap v357: 97 pgs: 97 active+clean; 5.1 GiB data, 15 GiB used, 1.0 TiB / 1.0 TiB avail; 743 KiB/s rd, 437 MiB/s wr, 357 op/s 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.286 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:34 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:35.932 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:36.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:36.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:36.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:35 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:36.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:36.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:36.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:35 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:01:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:01:36.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:35 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:01:37.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.259+0000 7f205bdfa640 1 -- 172.21.15.16:0/1256773150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2054102a60 msgr2=0x7f2054102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 --2- 172.21.15.16:0/1256773150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2054102a60 0x7f2054102e70 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2050009900 tx=0x7f205002a410 comp rx=0 tx=0).stop 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 -- 172.21.15.16:0/1256773150 shutdown_connections 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 --2- 172.21.15.16:0/1256773150 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f20541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 --2- 172.21.15.16:0/1256773150 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 0x7f205410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 --2- 172.21.15.16:0/1256773150 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2054102a60 0x7f2054102e70 secure :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2050009900 tx=0x7f205002a410 comp rx=0 tx=0).stop 2024-09-21T03:01:37.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.260+0000 7f205bdfa640 1 -- 172.21.15.16:0/1256773150 >> 172.21.15.16:0/1256773150 conn(0x7f2054079730 msgr2=0x7f2054079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.264 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:37 smithi016 ceph-mon[31253]: pgmap v358: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 670 MiB/s wr, 1.10k op/s 2024-09-21T03:01:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.261+0000 7f205bdfa640 1 -- 172.21.15.16:0/1256773150 shutdown_connections 2024-09-21T03:01:37.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.261+0000 7f205bdfa640 1 -- 172.21.15.16:0/1256773150 wait complete. 2024-09-21T03:01:37.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.264+0000 7f205bdfa640 1 Processor -- start 2024-09-21T03:01:37.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.265+0000 7f205bdfa640 1 -- start start 2024-09-21T03:01:37.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.266+0000 7f205bdfa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.266+0000 7f205bdfa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 0x7f205419ab40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.266+0000 7f205bdfa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205419fd20 0x7f20541a0130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.266+0000 7f205bdfa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f20541129b0 con 0x7f20541083a0 2024-09-21T03:01:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.267+0000 7f205bdfa640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2054112830 con 0x7f2054108cf0 2024-09-21T03:01:37.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.267+0000 7f205bdfa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2054112b30 con 0x7f205419fd20 2024-09-21T03:01:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.267+0000 7f2059b96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.268+0000 7f2059b96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37188/0 (socket says 172.21.15.16:37188) 2024-09-21T03:01:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.268+0000 7f2059b96640 1 -- 172.21.15.16:0/4257786489 learned_addr learned my addr 172.21.15.16:0/4257786489 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.269+0000 7f205a397640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205419fd20 0x7f20541a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.269+0000 7f2059395640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 0x7f205419ab40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.270+0000 7f2059b96640 1 -- 172.21.15.16:0/4257786489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205419fd20 msgr2=0x7f20541a0130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.270+0000 7f2059b96640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205419fd20 0x7f20541a0130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.270+0000 7f2059b96640 1 -- 172.21.15.16:0/4257786489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 msgr2=0x7f205419ab40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.270+0000 7f2059b96640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 0x7f205419ab40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.270+0000 7f2059b96640 1 -- 172.21.15.16:0/4257786489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20541a0900 con 0x7f20541083a0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.271+0000 7f2059b96640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f205002aa00 tx=0x7f2050003c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.272+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f205003f020 con 0x7f20541083a0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.272+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2054113150 con 0x7f20541083a0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.272+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2054113630 con 0x7f20541083a0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.273+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2050031d60 con 0x7f20541083a0 2024-09-21T03:01:37.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.273+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2050035070 con 0x7f20541083a0 2024-09-21T03:01:37.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.275+0000 7f2048ff9640 1 -- 172.21.15.16:0/4257786489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f203c005290 con 0x7f20541083a0 2024-09-21T03:01:37.292 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.287+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2050042020 con 0x7f20541083a0 2024-09-21T03:01:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.290+0000 7f204affd640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 0x7f20400b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.290+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2050107fb0 con 0x7f20541083a0 2024-09-21T03:01:37.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.291+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f2050049030 con 0x7f20541083a0 2024-09-21T03:01:37.295 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.293+0000 7f2059395640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 0x7f20400b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:37.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.294+0000 7f2059395640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 0x7f20400b8c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2054079e80 tx=0x7f2044005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:37 smithi018 ceph-mon[34587]: pgmap v358: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 670 MiB/s wr, 1.10k op/s 2024-09-21T03:01:37.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:37 smithi123 ceph-mon[34645]: pgmap v358: 97 pgs: 97 active+clean; 7.1 GiB data, 21 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 670 MiB/s wr, 1.10k op/s 2024-09-21T03:01:37.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.844+0000 7f2048ff9640 1 -- 172.21.15.16:0/4257786489 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f203c005580 con 0x7f20541083a0 2024-09-21T03:01:37.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.848+0000 7f204affd640 1 -- 172.21.15.16:0/4257786489 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f20500ce4c0 con 0x7f20541083a0 2024-09-21T03:01:37.851 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:01:37.852 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:01:37.852 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:01:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.854+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 msgr2=0x7f20400b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.854+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 0x7f20400b8c90 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2054079e80 tx=0x7f2044005f90 comp rx=0 tx=0).stop 2024-09-21T03:01:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.854+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 msgr2=0x7f205419a5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:37.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.854+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f205002aa00 tx=0x7f2050003c80 comp rx=0 tx=0).stop 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 shutdown_connections 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f20400b6780 0x7f20400b8c90 secure :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f2054079e80 tx=0x7f2044005f90 comp rx=0 tx=0).stop 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f205419fd20 0x7f20541a0130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2054108cf0 0x7f205419ab40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 --2- 172.21.15.16:0/4257786489 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f20541083a0 0x7f205419a5a0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:37.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.856+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 >> 172.21.15.16:0/4257786489 conn(0x7f2054079730 msgr2=0x7f2054100620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:37.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.860+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 shutdown_connections 2024-09-21T03:01:37.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:37.860+0000 7f205bdfa640 1 -- 172.21.15.16:0/4257786489 wait complete. 2024-09-21T03:01:38.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:38 smithi016 ceph-mon[31253]: pgmap v359: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 653 MiB/s wr, 1.13k op/s 2024-09-21T03:01:38.109 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:38 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/4257786489' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:38 smithi018 ceph-mon[34587]: pgmap v359: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 653 MiB/s wr, 1.13k op/s 2024-09-21T03:01:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:38 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/4257786489' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:38 smithi123 ceph-mon[34645]: pgmap v359: 97 pgs: 97 active+clean; 7.5 GiB data, 22 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 653 MiB/s wr, 1.13k op/s 2024-09-21T03:01:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:38 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/4257786489' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:01:39.377 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:01:39.922 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:40.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:40 smithi018 ceph-mon[34587]: pgmap v360: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 712 MiB/s wr, 1.19k op/s 2024-09-21T03:01:40.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:40 smithi123 ceph-mon[34645]: pgmap v360: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 712 MiB/s wr, 1.19k op/s 2024-09-21T03:01:40.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:40 smithi016 ceph-mon[31253]: pgmap v360: 97 pgs: 97 active+clean; 8.4 GiB data, 25 GiB used, 1.0 TiB / 1.0 TiB avail; 969 KiB/s rd, 712 MiB/s wr, 1.19k op/s 2024-09-21T03:01:40.992 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:01:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.885+0000 7f279da34640 1 -- 172.21.15.16:0/64397413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279810de50 msgr2=0x7f27981102f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.885+0000 7f279da34640 1 --2- 172.21.15.16:0/64397413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279810de50 0x7f27981102f0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7f2790009ca0 tx=0x7f279002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 -- 172.21.15.16:0/64397413 shutdown_connections 2024-09-21T03:01:40.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 --2- 172.21.15.16:0/64397413 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f279810de50 0x7f27981102f0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 --2- 172.21.15.16:0/64397413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981052d0 0x7f279810d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 --2- 172.21.15.16:0/64397413 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f27981048a0 0x7f2798104cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 -- 172.21.15.16:0/64397413 >> 172.21.15.16:0/64397413 conn(0x7f2798079730 msgr2=0x7f2798079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 -- 172.21.15.16:0/64397413 shutdown_connections 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.886+0000 7f279da34640 1 -- 172.21.15.16:0/64397413 wait complete. 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 Processor -- start 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 -- start start 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27981052d0 0x7f279819a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 0x7f279819fc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.887+0000 7f279da34640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2798113a60 con 0x7f27981052d0 2024-09-21T03:01:40.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44970/0 (socket says 172.21.15.16:44970) 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f2796ffd640 1 -- 172.21.15.16:0/3487292457 learned_addr learned my addr 172.21.15.16:0/3487292457 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27977fe640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 0x7f279819fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27981138e0 con 0x7f279810de50 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2798113be0 con 0x7f27981048a0 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27967fc640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27981052d0 0x7f279819a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27977fe640 1 -- 172.21.15.16:0/3487292457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 msgr2=0x7f279819a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27977fe640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27977fe640 1 -- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27981052d0 msgr2=0x7f279819a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.888+0000 7f27977fe640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27981052d0 0x7f279819a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f27977fe640 1 -- 172.21.15.16:0/3487292457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27981a0380 con 0x7f279810de50 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f2796ffd640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f27977fe640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 0x7f279819fc50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f279002ade0 tx=0x7f27900319f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f279ca32640 1 -- 172.21.15.16:0/3487292457 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2790003bf0 con 0x7f279810de50 2024-09-21T03:01:40.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f279ca32640 1 -- 172.21.15.16:0/3487292457 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2790003d90 con 0x7f279810de50 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27981a0610 con 0x7f279810de50 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f279ca32640 1 -- 172.21.15.16:0/3487292457 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f279002fbd0 con 0x7f279810de50 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.889+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27981a6dd0 con 0x7f279810de50 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.893+0000 7f27a3fa8740 1 Processor -- start 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.895+0000 7f279ca32640 1 -- 172.21.15.16:0/3487292457 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f279002fd70 con 0x7f279810de50 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.895+0000 7f27a3fa8740 1 -- start start 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5ea1b0 0x56459c9babe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9bb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56459c930ea0 con 0x56459c613070 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56459c9202d0 con 0x56459c5ea1b0 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56459c9240e0 con 0x56459c5f2390 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37220/0 (socket says 172.21.15.16:37220) 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.896+0000 7f27967fc640 1 -- 172.21.15.16:0/3452283087 learned_addr learned my addr 172.21.15.16:0/3452283087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 -- 172.21.15.16:0/3452283087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 msgr2=0x56459c9bb120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9bb120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 -- 172.21.15.16:0/3452283087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5ea1b0 msgr2=0x56459c9babe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5ea1b0 0x56459c9babe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 -- 172.21.15.16:0/3452283087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459c67e650 con 0x56459c613070 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.897+0000 7f27967fc640 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f278000bc40 tx=0x7f27800107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1caea115a6e9f67 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.898+0000 7f279ca32640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 0x7f27840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.898+0000 7f278dffb640 1 -- 172.21.15.16:0/3452283087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780015850 con 0x56459c613070 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.898+0000 7f2796ffd640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 0x7f27840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.898+0000 7f278dffb640 1 -- 172.21.15.16:0/3452283087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780005020 con 0x56459c613070 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.899+0000 7f279ca32640 1 -- 172.21.15.16:0/3487292457 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2790108980 con 0x7f279810de50 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.899+0000 7f278dffb640 1 -- 172.21.15.16:0/3452283087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780004b10 con 0x56459c613070 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 -- 172.21.15.16:0/3452283087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 msgr2=0x56459c9ba6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:40.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f278000bc40 tx=0x7f27800107a0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f2796ffd640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 0x7f27840b8c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f278800b6c0 tx=0x7f278800b650 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 -- 172.21.15.16:0/3452283087 shutdown_connections 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9bb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5ea1b0 0x56459c9babe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/3452283087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c613070 0x56459c9ba6a0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.900+0000 7f27a3fa8740 1 -- 172.21.15.16:0/3452283087 >> 172.21.15.16:0/3452283087 conn(0x56459c607460 msgr2=0x56459c9b8590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.901+0000 7f27a3fa8740 1 -- 172.21.15.16:0/3452283087 shutdown_connections 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.901+0000 7f27a3fa8740 1 -- 172.21.15.16:0/3452283087 wait complete. 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.902+0000 7f27a3fa8740 1 -- start start 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.902+0000 7f27a3fa8740 1 Processor -- start 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9c6470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9daa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56459c781700 con 0x56459c5ea1b0 2024-09-21T03:01:40.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56459c8bb160 con 0x56459c5d19f0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56459c9c8ed0 con 0x56459c5f2390 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f27a3fa8740 10 client.0 _ll_register_callbacks cb 0x56459c8d49e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37232/0 (socket says 172.21.15.16:37232) 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.903+0000 7f2796ffd640 1 -- 172.21.15.16:0/4234540698 learned_addr learned my addr 172.21.15.16:0/4234540698 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f27967fc640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9c6470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f27977fe640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9daa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2796ffd640 1 -- 172.21.15.16:0/4234540698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 msgr2=0x56459c9c6470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2796ffd640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9c6470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2796ffd640 1 -- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 msgr2=0x56459c9daa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2796ffd640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9daa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2796ffd640 1 -- 172.21.15.16:0/4234540698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459c8e8c90 con 0x56459c5ea1b0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.904+0000 7f2776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:01:40.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.905+0000 7f2796ffd640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f27880119d0 tx=0x7f278800b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e55f55e496867ef7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.905+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278801cc50 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.905+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2780005020 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.905+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f278801cdf0 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.906+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2788004020 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.906+0000 7f27a3fa8740 10 client.15594 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.906+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56459c9dd650 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780005020 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2788026430 con 0x56459c5ea1b0 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f27a3fa8740 10 client.15594 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f27a3fa8740 10 client.15594 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f27a3fa8740 20 client.15594 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.908+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459c5a6da0 0x56459c9e0b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.909+0000 7f27a3fa8740 4 client.15594 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:41.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.909+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56459c932df0 con 0x56459c5a6da0 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.910+0000 7f27967fc640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459c5a6da0 0x56459c9e0b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.913+0000 7f27967fc640 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459c5a6da0 0x56459c9e0b30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a7406c36539396 server_cookie=a93716c31a951821 in_seq=0 out_seq=0 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.913+0000 7f2776ffd640 10 client.15594 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.914+0000 7f2776ffd640 1 -- 172.21.15.16:0/4234540698 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56459c932df0 con 0x56459c5a6da0 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.914+0000 7f2776ffd640 10 client.15594 handle_command_reply: tid=0 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.914+0000 7f27a3fa8740 1 client.15594 shutdown 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 msgr2=0x56459c9c5ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f27880119d0 tx=0x7f278800b410 comp rx=0 tx=0).stop 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 shutdown_connections 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459c5a6da0 0x56459c9e0b30 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9daa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5f2390 0x56459c9c6470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/4234540698 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5ea1b0 0x56459c9c5ef0 secure :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f27880119d0 tx=0x7f278800b410 comp rx=0 tx=0).stop 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f2776ffd640 0 client.15594 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.915+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 >> 172.21.15.16:0/4234540698 conn(0x56459c607460 msgr2=0x56459c9bd880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:41.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.916+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 shutdown_connections 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.916+0000 7f27a3fa8740 1 -- 172.21.15.16:0/4234540698 wait complete. 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.917+0000 7f27a3fa8740 20 client.15594 trim_cache size 0 max 16384 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.951+0000 7f27a3fa8740 1 Processor -- start 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.952+0000 7f27a3fa8740 1 -- start start 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.952+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.953+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9c30e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.953+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9c3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27981ab480 con 0x56459c5f2390 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f2796ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44976/0 (socket says 172.21.15.16:44976) 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f2796ffd640 1 -- 172.21.15.16:0/367291956 learned_addr learned my addr 172.21.15.16:0/367291956 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f27967fc640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9c30e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f27977fe640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9c3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27981add00 con 0x56459c5d19f0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.954+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27981abaf0 con 0x56459c5a6da0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.955+0000 7f27967fc640 1 -- 172.21.15.16:0/367291956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 msgr2=0x56459c9be160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.955+0000 7f27967fc640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.955+0000 7f27967fc640 1 -- 172.21.15.16:0/367291956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 msgr2=0x56459c9c3620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.956+0000 7f27967fc640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9c3620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.956+0000 7f27967fc640 1 -- 172.21.15.16:0/367291956 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56459c9c43f0 con 0x56459c5d19f0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.956+0000 7f2796ffd640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.957+0000 7f27977fe640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.957+0000 7f27967fc640 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9c30e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2780018000 tx=0x7f278000b390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ba42feae1fb6886 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.960+0000 7f278d7fa640 1 -- 172.21.15.16:0/367291956 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f278000b820 con 0x56459c5d19f0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.960+0000 7f278d7fa640 1 -- 172.21.15.16:0/367291956 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f27800044f0 con 0x56459c5d19f0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.960+0000 7f278d7fa640 1 -- 172.21.15.16:0/367291956 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780029470 con 0x56459c5d19f0 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 msgr2=0x56459c9c30e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:41.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9c30e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2780018000 tx=0x7f278000b390 comp rx=0 tx=0).stop 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 shutdown_connections 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9c3620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5d19f0 0x56459c9c30e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/367291956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5a6da0 0x56459c9be160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.961+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 >> 172.21.15.16:0/367291956 conn(0x56459c607460 msgr2=0x56459c9c4770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.962+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 shutdown_connections 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.962+0000 7f27a3fa8740 1 -- 172.21.15.16:0/367291956 wait complete. 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.963+0000 7f27a3fa8740 1 -- start start 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.964+0000 7f27a3fa8740 1 Processor -- start 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.965+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5a6da0 0x56459c9d73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.965+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 0x56459c9d3890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27a3fa8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9d3fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f27981ac1c0 con 0x56459c5f2390 2024-09-21T03:01:41.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27981ad630 con 0x56459c5a6da0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27a3fa8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f27981ac890 con 0x56459c5d19f0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9d3fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.966+0000 7f27a3fa8740 10 client.0 _ll_register_callbacks cb 0x7f278800bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.967+0000 7f27977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9d3fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37248/0 (socket says 172.21.15.16:37248) 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.967+0000 7f27977fe640 1 -- 172.21.15.16:0/2712466602 learned_addr learned my addr 172.21.15.16:0/2712466602 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.967+0000 7f27967fc640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 0x56459c9d3890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.967+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5a6da0 0x56459c9d73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.968+0000 7f27967fc640 1 -- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5a6da0 msgr2=0x56459c9d73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.968+0000 7f27967fc640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5a6da0 0x56459c9d73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.968+0000 7f27967fc640 1 -- 172.21.15.16:0/2712466602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 msgr2=0x56459c9d3fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.968+0000 7f27967fc640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9d3fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.968+0000 7f27967fc640 1 -- 172.21.15.16:0/2712466602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56459ca4b540 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.969+0000 7f27967fc640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 0x56459c9d3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2780010ca0 tx=0x7f2780005900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d25850effef7279 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.969+0000 7f2776ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.969+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2780003860 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.969+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f2780005020 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.970+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f27800044f0 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.971+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2780005220 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.971+0000 7f27a3fa8740 10 client.25240 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.971+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56459ca4c660 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.972+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2780004c50 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.974+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f27800047a0 con 0x56459c5d19f0 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.974+0000 7f27a3fa8740 10 client.25240 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.974+0000 7f27a3fa8740 10 client.25240 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.974+0000 7f27a3fa8740 20 client.25240 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:41.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.974+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459ca4ca90 0x56459ca4eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.975+0000 7f27a3fa8740 4 client.25240 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.975+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x7f27800044f0 con 0x56459ca4ca90 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.976+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459ca4ca90 0x56459ca4eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.978+0000 7f2776ffd640 10 client.25240 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.978+0000 7f2796ffd640 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459ca4ca90 0x56459ca4eeb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8fb0efa413c89057 server_cookie=5b1d2505c06029cb in_seq=0 out_seq=0 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.979+0000 7f2776ffd640 1 -- 172.21.15.16:0/2712466602 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f27800044f0 con 0x56459ca4ca90 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.979+0000 7f2776ffd640 10 client.25240 handle_command_reply: tid=0 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.979+0000 7f27a3fa8740 1 client.25240 shutdown 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.981+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 msgr2=0x56459c9d3890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.981+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 0x56459c9d3890 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f2780010ca0 tx=0x7f2780005900 comp rx=0 tx=0).stop 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.982+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 shutdown_connections 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.982+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56459ca4ca90 0x56459ca4eeb0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.982+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56459c5f2390 0x56459c9d3fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.982+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56459c5d19f0 0x56459c9d3890 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.983+0000 7f27a3fa8740 1 --2- 172.21.15.16:0/2712466602 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56459c5a6da0 0x56459c9d73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.983+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 >> 172.21.15.16:0/2712466602 conn(0x56459c607460 msgr2=0x56459c9c4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.983+0000 7f2776ffd640 0 client.25240 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.983+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 shutdown_connections 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.984+0000 7f27a3fa8740 1 -- 172.21.15.16:0/2712466602 wait complete. 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.984+0000 7f27a3fa8740 20 client.25240 trim_cache size 0 max 16384 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.987+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 msgr2=0x7f27840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.987+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 0x7f27840b8c10 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f278800b6c0 tx=0x7f278800b650 comp rx=0 tx=0).stop 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.987+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 msgr2=0x7f279819fc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:41.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.987+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 0x7f279819fc50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f279002ade0 tx=0x7f27900319f0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 shutdown_connections 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f27840b6700 0x7f27840b8c10 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f27981052d0 0x7f279819a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f279810de50 0x7f279819fc50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 --2- 172.21.15.16:0/3487292457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f27981048a0 0x7f279819a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.991+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 >> 172.21.15.16:0/3487292457 conn(0x7f2798079730 msgr2=0x7f2798100890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.992+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 shutdown_connections 2024-09-21T03:01:41.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:40.992+0000 7f279da34640 1 -- 172.21.15.16:0/3487292457 wait complete. 2024-09-21T03:01:42.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:42 smithi018 ceph-mon[34587]: pgmap v361: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 770 MiB/s wr, 1.29k op/s 2024-09-21T03:01:42.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:42 smithi123 ceph-mon[34645]: pgmap v361: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 770 MiB/s wr, 1.29k op/s 2024-09-21T03:01:42.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:42 smithi016 ceph-mon[31253]: pgmap v361: 97 pgs: 97 active+clean; 9.9 GiB data, 30 GiB used, 1.0 TiB / 1.0 TiB avail; 1.4 MiB/s rd, 770 MiB/s wr, 1.29k op/s 2024-09-21T03:01:43.062 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:01:43.062 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:01:43.628 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:44.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 -- 172.21.15.16:0/3539938790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978109fc0 msgr2=0x7fc97810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/3539938790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978109fc0 0x7fc97810a3d0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7fc968009900 tx=0x7fc96802a410 comp rx=0 tx=0).stop 2024-09-21T03:01:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 -- 172.21.15.16:0/3539938790 shutdown_connections 2024-09-21T03:01:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/3539938790 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc978110250 0x7fc978116a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/3539938790 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc97810f900 0x7fc97810fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/3539938790 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978109fc0 0x7fc97810a3d0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.431+0000 7fc97e4c3640 1 -- 172.21.15.16:0/3539938790 >> 172.21.15.16:0/3539938790 conn(0x7fc9780794a0 msgr2=0x7fc9781081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.432+0000 7fc97e4c3640 1 -- 172.21.15.16:0/3539938790 shutdown_connections 2024-09-21T03:01:44.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.432+0000 7fc97e4c3640 1 -- 172.21.15.16:0/3539938790 wait complete. 2024-09-21T03:01:44.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.434+0000 7fc97e4c3640 1 Processor -- start 2024-09-21T03:01:44.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.434+0000 7fc97e4c3640 1 -- start start 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.435+0000 7fc97e4c3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc97e4c3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc97810f900 0x7fc9781a0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc977fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc977fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44292/0 (socket says 172.21.15.16:44292) 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc97e4c3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978110250 0x7fc9781a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc977fff640 1 -- 172.21.15.16:0/982335097 learned_addr learned my addr 172.21.15.16:0/982335097 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc978119fa0 con 0x7fc978110250 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc97ca60640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978110250 0x7fc9781a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.436+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc978119e20 con 0x7fc97810f900 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc97811a120 con 0x7fc978109fc0 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc977fff640 1 -- 172.21.15.16:0/982335097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc97810f900 msgr2=0x7fc9781a0ed0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc977fff640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc97810f900 0x7fc9781a0ed0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc977fff640 1 -- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978110250 msgr2=0x7fc9781a6170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc977fff640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978110250 0x7fc9781a6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.437+0000 7fc977fff640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9781a67e0 con 0x7fc978109fc0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.438+0000 7fc977fff640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc968005ab0 tx=0x7fc968003d50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.438+0000 7fc9757fa640 1 -- 172.21.15.16:0/982335097 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96803f070 con 0x7fc978109fc0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.438+0000 7fc9757fa640 1 -- 172.21.15.16:0/982335097 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc968031d60 con 0x7fc978109fc0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.438+0000 7fc9757fa640 1 -- 172.21.15.16:0/982335097 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968035030 con 0x7fc978109fc0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.438+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9781a6a70 con 0x7fc978109fc0 2024-09-21T03:01:44.442 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.439+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9781a6ec0 con 0x7fc978109fc0 2024-09-21T03:01:44.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.444+0000 7fc984a37740 1 Processor -- start 2024-09-21T03:01:44.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.444+0000 7fc984a37740 1 -- start start 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.445+0000 7fc9757fa640 1 -- 172.21.15.16:0/982335097 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc968042070 con 0x7fc978109fc0 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.447+0000 7fc9757fa640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 0x7fc9600b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc97ca60640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 0x7fc9600b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc9757fa640 1 -- 172.21.15.16:0/982335097 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc968108130 con 0x7fc978109fc0 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f4f7610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4f8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f46de60 con 0x55ca6f150350 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f45d290 con 0x55ca6f12f670 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.448+0000 7fc984a37740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f4610a0 con 0x55ca6f127490 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44302/0 (socket says 172.21.15.16:44302) 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 -- 172.21.15.16:0/342904765 learned_addr learned my addr 172.21.15.16:0/342904765 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:44.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc97ca60640 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4f8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 -- 172.21.15.16:0/342904765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 msgr2=0x55ca6f4f8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4f8090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 -- 172.21.15.16:0/342904765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 msgr2=0x55ca6f4f7610 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f4f7610 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 -- 172.21.15.16:0/342904765 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6f1bb930 con 0x55ca6f127490 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.449+0000 7fc9777fe640 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0096f0 tx=0x7fc96c009bb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b75537e59a3f6215 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.450+0000 7fc95e7fc640 1 -- 172.21.15.16:0/342904765 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c015020 con 0x55ca6f127490 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.450+0000 7fc95e7fc640 1 -- 172.21.15.16:0/342904765 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96c00bd80 con 0x55ca6f127490 2024-09-21T03:01:44.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.450+0000 7fc95e7fc640 1 -- 172.21.15.16:0/342904765 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c010380 con 0x55ca6f127490 2024-09-21T03:01:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.451+0000 7fc984a37740 1 -- 172.21.15.16:0/342904765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 msgr2=0x55ca6f4f7b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.451+0000 7fc97ca60640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 0x7fc9600b8c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc970008ec0 tx=0x7fc9700117b0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.451+0000 7fc984a37740 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc96c0096f0 tx=0x7fc96c009bb0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 -- 172.21.15.16:0/342904765 shutdown_connections 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4f8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f127490 0x55ca6f4f7b50 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 --2- 172.21.15.16:0/342904765 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f4f7610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 -- 172.21.15.16:0/342904765 >> 172.21.15.16:0/342904765 conn(0x55ca6f144740 msgr2=0x55ca6f4f5500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 -- 172.21.15.16:0/342904765 shutdown_connections 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 -- 172.21.15.16:0/342904765 wait complete. 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.452+0000 7fc984a37740 1 -- start start 2024-09-21T03:01:44.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.453+0000 7fc984a37740 1 Processor -- start 2024-09-21T03:01:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.455 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f12f670 0x55ca6f5037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f50f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc977fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc977fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:35264/0 (socket says 172.21.15.16:35264) 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f2be890 con 0x55ca6f150350 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc977fff640 1 -- 172.21.15.16:0/3569140472 learned_addr learned my addr 172.21.15.16:0/3569140472 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f3f7d40 con 0x55ca6f127490 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca6f505e40 con 0x55ca6f12f670 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc9777fe640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f12f670 0x55ca6f5037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc984a37740 10 client.0 _ll_register_callbacks cb 0x55ca6f45bc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:44.456 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc97ca60640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f50f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc977fff640 1 -- 172.21.15.16:0/3569140472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f12f670 msgr2=0x55ca6f5037e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.454+0000 7fc977fff640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f12f670 0x55ca6f5037e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc977fff640 1 -- 172.21.15.16:0/3569140472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 msgr2=0x55ca6f50f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc977fff640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f50f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc977fff640 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6f50f770 con 0x55ca6f127490 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc977fff640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cdcc0 tx=0x7fc9680c9e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aae8d1eb0abaab6e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc9577fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:01:44.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc968049070 con 0x55ca6f127490 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.455+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc9680d7750 con 0x55ca6f127490 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.456+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96c00bd80 con 0x55ca6f127490 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.456+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc9680cdad0 con 0x55ca6f127490 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.456+0000 7fc984a37740 10 client.25313 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.456+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ca6f512060 con 0x55ca6f127490 2024-09-21T03:01:44.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96803be50 con 0x55ca6f127490 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc9680d0060 con 0x55ca6f127490 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 10 client.25313 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 10 client.25313 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 20 client.25313 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f10ecd0 0x55ca6f515540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 4 client.25313 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fc96c00bd80 con 0x55ca6f10ecd0 2024-09-21T03:01:44.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.457+0000 7fc9777fe640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f10ecd0 0x55ca6f515540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.459+0000 7fc9777fe640 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f10ecd0 0x55ca6f515540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8dcdfafce4fc6122 server_cookie=8e265e3eb95396eb in_seq=0 out_seq=0 2024-09-21T03:01:44.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.459+0000 7fc9577fe640 10 client.25313 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc9577fe640 1 -- 172.21.15.16:0/3569140472 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fc96c00bd80 con 0x55ca6f10ecd0 2024-09-21T03:01:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc9577fe640 10 client.25313 handle_command_reply: tid=0 2024-09-21T03:01:44.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 client.25313 shutdown 2024-09-21T03:01:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 msgr2=0x55ca6f503240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fc9680cdcc0 tx=0x7fc9680c9e60 comp rx=0 tx=0).stop 2024-09-21T03:01:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 shutdown_connections 2024-09-21T03:01:44.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f10ecd0 0x55ca6f515540 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f150350 0x55ca6f50f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f12f670 0x55ca6f5037e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 --2- 172.21.15.16:0/3569140472 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f127490 0x55ca6f503240 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 >> 172.21.15.16:0/3569140472 conn(0x55ca6f144740 msgr2=0x55ca6f4fa7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.460+0000 7fc9577fe640 0 client.25313 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.461+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 shutdown_connections 2024-09-21T03:01:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.461+0000 7fc984a37740 1 -- 172.21.15.16:0/3569140472 wait complete. 2024-09-21T03:01:44.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.461+0000 7fc984a37740 20 client.25313 trim_cache size 0 max 16384 2024-09-21T03:01:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.493+0000 7fc984a37740 1 Processor -- start 2024-09-21T03:01:44.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.493+0000 7fc984a37740 1 -- start start 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f4fc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4fcc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc977fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f4fc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc977fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f4fc720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33610/0 (socket says 172.21.15.16:33610) 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc977fff640 1 -- 172.21.15.16:0/510304624 learned_addr learned my addr 172.21.15.16:0/510304624 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f4f4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b3a10 con 0x55ca6f127490 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b6290 con 0x55ca6f12f670 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b4080 con 0x55ca6f150350 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc9777fe640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4fcc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.496 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.494+0000 7fc97ca60640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f4f4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 -- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 msgr2=0x55ca6f4f4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f4f4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 -- 172.21.15.16:0/510304624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 msgr2=0x55ca6f4fc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f4fc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.497 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 -- 172.21.15.16:0/510304624 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca6f4fe8d0 con 0x55ca6f12f670 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc97ca60640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f4f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc9777fe640 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4fcc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc96c004810 tx=0x7fc96c01c040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42e8b136e9f3687b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc95dffb640 1 -- 172.21.15.16:0/510304624 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c015040 con 0x55ca6f12f670 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc95dffb640 1 -- 172.21.15.16:0/510304624 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96c0118d0 con 0x55ca6f12f670 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.495+0000 7fc95dffb640 1 -- 172.21.15.16:0/510304624 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c0268d0 con 0x55ca6f12f670 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 msgr2=0x55ca6f4fcc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4fcc60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fc96c004810 tx=0x7fc96c01c040 comp rx=0 tx=0).stop 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 shutdown_connections 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f4f4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f4fcc60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 --2- 172.21.15.16:0/510304624 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f4fc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.496+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 >> 172.21.15.16:0/510304624 conn(0x55ca6f144740 msgr2=0x55ca6f5018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.497+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 shutdown_connections 2024-09-21T03:01:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.497+0000 7fc984a37740 1 -- 172.21.15.16:0/510304624 wait complete. 2024-09-21T03:01:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.497+0000 7fc984a37740 1 -- start start 2024-09-21T03:01:44.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.497+0000 7fc984a37740 1 Processor -- start 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f507450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f57ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b4750 con 0x55ca6f127490 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b5bc0 con 0x55ca6f12f670 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc9781b4e20 con 0x55ca6f150350 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc984a37740 10 client.0 _ll_register_callbacks cb 0x7fc96803b460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:35280/0 (socket says 172.21.15.16:35280) 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 -- 172.21.15.16:0/26341244 learned_addr learned my addr 172.21.15.16:0/26341244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc97ca60640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f57ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 -- 172.21.15.16:0/26341244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 msgr2=0x55ca6f57ef00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f57ef00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc977fff640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f507450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.498+0000 7fc9777fe640 1 -- 172.21.15.16:0/26341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 msgr2=0x55ca6f507450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9777fe640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f507450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9777fe640 1 -- 172.21.15.16:0/26341244 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca6f57f440 con 0x55ca6f12f670 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc977fff640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f507450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9777fe640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c00b5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=90ae0d28e30e28cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9577fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc96c011a80 con 0x55ca6f12f670 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc96c011620 con 0x55ca6f12f670 2024-09-21T03:01:44.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.499+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc96c0118d0 con 0x55ca6f12f670 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc96c006b60 con 0x55ca6f12f670 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 10 client.25319 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ca6f5805b0 con 0x55ca6f12f670 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc96c0118d0 con 0x55ca6f12f670 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc96c035070 con 0x55ca6f12f670 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 10 client.25319 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 10 client.25319 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 20 client.25319 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f0e4080 0x55ca6f582ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 4 client.25319 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.500+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55ca6f46fdb0 con 0x55ca6f0e4080 2024-09-21T03:01:44.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.501+0000 7fc977fff640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f0e4080 0x55ca6f582ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.501+0000 7fc9577fe640 10 client.25319 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.501+0000 7fc977fff640 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f0e4080 0x55ca6f582ac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b32f10c4e142533 server_cookie=1270ff73276204cf in_seq=0 out_seq=0 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc9577fe640 1 -- 172.21.15.16:0/26341244 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55ca6f46fdb0 con 0x55ca6f0e4080 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc9577fe640 10 client.25319 handle_command_reply: tid=0 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc984a37740 1 client.25319 shutdown 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 msgr2=0x55ca6f5079b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc96c006fd0 tx=0x7fc96c00b5f0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 shutdown_connections 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.503+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca6f0e4080 0x55ca6f582ac0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc9577fe640 0 client.25319 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca6f150350 0x55ca6f57ef00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca6f12f670 0x55ca6f5079b0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 --2- 172.21.15.16:0/26341244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca6f127490 0x55ca6f507450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 >> 172.21.15.16:0/26341244 conn(0x55ca6f144740 msgr2=0x55ca6f5044b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 shutdown_connections 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 1 -- 172.21.15.16:0/26341244 wait complete. 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.504+0000 7fc984a37740 20 client.25319 trim_cache size 0 max 16384 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "15fd7ce4-15fb-4adf-a38a-09586ce5da82", 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:01:44.507 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 msgr2=0x7fc9600b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 0x7fc9600b8c40 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc970008ec0 tx=0x7fc9700117b0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 msgr2=0x7fc9781a0930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc968005ab0 tx=0x7fc968003d50 comp rx=0 tx=0).stop 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 shutdown_connections 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc9600b6730 0x7fc9600b8c40 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc978110250 0x7fc9781a6170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc97810f900 0x7fc9781a0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 --2- 172.21.15.16:0/982335097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc978109fc0 0x7fc9781a0930 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 >> 172.21.15.16:0/982335097 conn(0x7fc9780794a0 msgr2=0x7fc978108680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 shutdown_connections 2024-09-21T03:01:44.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:44.505+0000 7fc97e4c3640 1 -- 172.21.15.16:0/982335097 wait complete. 2024-09-21T03:01:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:44 smithi123 ceph-mon[34645]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 605 MiB/s wr, 1.20k op/s 2024-09-21T03:01:45.026 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:44 smithi016 ceph-mon[31253]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 605 MiB/s wr, 1.20k op/s 2024-09-21T03:01:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:44 smithi018 ceph-mon[34587]: pgmap v362: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 792 KiB/s rd, 605 MiB/s wr, 1.20k op/s 2024-09-21T03:01:45.569 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:01:45.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:45 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:01:45.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:45 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:45.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:45 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:01:45.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:45 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:45 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:01:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:45 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:45 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:01:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:45 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:46.101 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:01:46.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:45 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:01:46.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:45 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:46.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:45 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:01:46.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:45 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:01:46.758 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:46 smithi016 ceph-mon[31253]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 MiB/s wr, 1.85k op/s 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 -- 172.21.15.16:0/1200377457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe158108480 msgr2=0x7fe15810ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 --2- 172.21.15.16:0/1200377457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe158108480 0x7fe15810ecb0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7fe150009ca0 tx=0x7fe15002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 -- 172.21.15.16:0/1200377457 shutdown_connections 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 --2- 172.21.15.16:0/1200377457 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe158108480 0x7fe15810ecb0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 --2- 172.21.15.16:0/1200377457 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe158107f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 --2- 172.21.15.16:0/1200377457 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe1581021f0 0x7fe158102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 -- 172.21.15.16:0/1200377457 >> 172.21.15.16:0/1200377457 conn(0x7fe158077c60 msgr2=0x7fe1580780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 -- 172.21.15.16:0/1200377457 shutdown_connections 2024-09-21T03:01:46.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.763+0000 7fe1602ea640 1 -- 172.21.15.16:0/1200377457 wait complete. 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.764+0000 7fe1602ea640 1 Processor -- start 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.764+0000 7fe1602ea640 1 -- start start 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.764+0000 7fe1602ea640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 0x7fe158199310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe1602ea640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe1602ea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 0x7fe15819eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe1602ea640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581129c0 con 0x7fe1581021f0 2024-09-21T03:01:46.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe1602ea640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe158112840 con 0x7fe158107b30 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe1602ea640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe158112b40 con 0x7fe158108480 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15e086640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 0x7fe158199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:35298/0 (socket says 172.21.15.16:35298) 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 -- 172.21.15.16:0/3904253724 learned_addr learned my addr 172.21.15.16:0/3904253724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15e887640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 0x7fe15819eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 -- 172.21.15.16:0/3904253724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 msgr2=0x7fe15819eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 0x7fe15819eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 -- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 msgr2=0x7fe158199310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 0x7fe158199310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15d885640 1 -- 172.21.15.16:0/3904253724 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe15819f280 con 0x7fe158107b30 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15e887640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 0x7fe15819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.765+0000 7fe15e086640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 0x7fe158199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe15d885640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e970 tx=0x7fe14800ee30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe14f7fe640 1 -- 172.21.15.16:0/3904253724 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe14800cd80 con 0x7fe158107b30 2024-09-21T03:01:46.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe14f7fe640 1 -- 172.21.15.16:0/3904253724 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1480035f0 con 0x7fe158107b30 2024-09-21T03:01:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe15819f570 con 0x7fe158107b30 2024-09-21T03:01:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe14f7fe640 1 -- 172.21.15.16:0/3904253724 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe148010690 con 0x7fe158107b30 2024-09-21T03:01:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.766+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a5cd0 con 0x7fe158107b30 2024-09-21T03:01:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.768+0000 7fe16685e740 1 Processor -- start 2024-09-21T03:01:46.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 -- start start 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d8727b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4aa390 0x562a4d872cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a4d7e8e50 con 0x562a4d4aa390 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562a4d7d8280 con 0x562a4d4cb070 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe16685e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562a4d7dc090 con 0x562a4d4a21b0 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe15e887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe15e086640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d8727b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe15e887640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:35304/0 (socket says 172.21.15.16:35304) 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.769+0000 7fe15e887640 1 -- 172.21.15.16:0/1224532501 learned_addr learned my addr 172.21.15.16:0/1224532501 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 -- 172.21.15.16:0/1224532501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 msgr2=0x562a4d8727b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d8727b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 -- 172.21.15.16:0/1224532501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4aa390 msgr2=0x562a4d872cf0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:01:46.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4aa390 0x562a4d872cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 -- 172.21.15.16:0/1224532501 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4d536650 con 0x562a4d4cb070 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe15e887640 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe150033040 tx=0x7fe15002f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e6933e0eff555bfc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe14cd9a640 1 -- 172.21.15.16:0/1224532501 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150031650 con 0x562a4d4cb070 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe14cd9a640 1 -- 172.21.15.16:0/1224532501 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe150036040 con 0x562a4d4cb070 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.770+0000 7fe14cd9a640 1 -- 172.21.15.16:0/1224532501 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150004560 con 0x562a4d4cb070 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 -- 172.21.15.16:0/1224532501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 msgr2=0x562a4d872270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe150033040 tx=0x7fe15002f3c0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 -- 172.21.15.16:0/1224532501 shutdown_connections 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4aa390 0x562a4d872cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d8727b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 --2- 172.21.15.16:0/1224532501 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d872270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.771+0000 7fe16685e740 1 -- 172.21.15.16:0/1224532501 >> 172.21.15.16:0/1224532501 conn(0x562a4d4bf460 msgr2=0x562a4d870160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.772+0000 7fe14f7fe640 1 -- 172.21.15.16:0/3904253724 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe148004020 con 0x7fe158107b30 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.772+0000 7fe16685e740 1 -- 172.21.15.16:0/1224532501 shutdown_connections 2024-09-21T03:01:46.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.772+0000 7fe16685e740 1 -- 172.21.15.16:0/1224532501 wait complete. 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.772+0000 7fe16685e740 1 -- start start 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.772+0000 7fe16685e740 1 Processor -- start 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe14f7fe640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 0x7fe1440b8b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d87fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d8800f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a4d639700 con 0x562a4d4899f0 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562a4d773110 con 0x562a4d4cb070 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562a4d8829d0 con 0x562a4d4a21b0 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe15e887640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 0x7fe1440b8b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe14f7fe640 1 -- 172.21.15.16:0/3904253724 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe1480e7880 con 0x7fe158107b30 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe16685e740 10 client.0 _ll_register_callbacks cb 0x562a4d78c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe15e887640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe15e887640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe15e887640 1 -- 172.21.15.16:0/2434227823 learned_addr learned my addr 172.21.15.16:0/2434227823 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:46.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.773+0000 7fe15d885640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d8800f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15e086640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d87fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 -- 172.21.15.16:0/2434227823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 msgr2=0x562a4d87fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d87fbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 -- 172.21.15.16:0/2434227823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 msgr2=0x562a4d88bd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 -- 172.21.15.16:0/2434227823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4d7a0c40 con 0x562a4d4cb070 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15e086640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d87fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15e887640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:46.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15e887640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 0x7fe1440b8b20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe150009ca0 tx=0x7fe1500040e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe15d885640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d8800f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1480e4ff0 tx=0x7fe1480acf20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2d2b9c9a5fc154b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.774+0000 7fe141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1480d4060 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe150036040 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe148003aa0 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe148003ca0 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe16685e740 10 client.25331 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562a4d8799f0 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe150036040 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fe1480a9240 con 0x562a4d4cb070 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.775+0000 7fe16685e740 10 client.25331 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe16685e740 10 client.25331 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe16685e740 20 client.25331 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d45eda0 0x562a4d879c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe16685e740 4 client.25331 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a4d7eada0 con 0x562a4d45eda0 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe15e086640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d45eda0 0x562a4d879c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe141ffb640 10 client.25331 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.776+0000 7fe15e086640 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d45eda0 0x562a4d879c80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=abd910f9c0729a6 server_cookie=80ebfa8a056d59ad in_seq=0 out_seq=0 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe141ffb640 1 -- 172.21.15.16:0/2434227823 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a4d7eada0 con 0x562a4d45eda0 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe141ffb640 10 client.25331 handle_command_reply: tid=0 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 client.25331 shutdown 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 msgr2=0x562a4d8800f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d8800f0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe1480e4ff0 tx=0x7fe1480acf20 comp rx=0 tx=0).stop 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 shutdown_connections 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d45eda0 0x562a4d879c80 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4899f0 0x562a4d88bd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d8800f0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 --2- 172.21.15.16:0/2434227823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4a21b0 0x562a4d87fbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 >> 172.21.15.16:0/2434227823 conn(0x562a4d4bf460 msgr2=0x562a4d871220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe141ffb640 0 client.25331 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 shutdown_connections 2024-09-21T03:01:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 1 -- 172.21.15.16:0/2434227823 wait complete. 2024-09-21T03:01:46.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.778+0000 7fe16685e740 20 client.25331 trim_cache size 0 max 16384 2024-09-21T03:01:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.800+0000 7fe16685e740 1 Processor -- start 2024-09-21T03:01:46.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.800+0000 7fe16685e740 1 -- start start 2024-09-21T03:01:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 0x562a4d878130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d87b3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ab350 con 0x562a4d4a21b0 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581adbd0 con 0x562a4d4cb070 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe16685e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ab9c0 con 0x562a4d4899f0 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe15e086640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe15e086640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44378/0 (socket says 172.21.15.16:44378) 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe15e086640 1 -- 172.21.15.16:0/439787598 learned_addr learned my addr 172.21.15.16:0/439787598 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe15e887640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d87b3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.801+0000 7fe15d885640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 0x562a4d878130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 -- 172.21.15.16:0/439787598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 msgr2=0x562a4d86ea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 -- 172.21.15.16:0/439787598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 msgr2=0x562a4d878130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 0x562a4d878130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 -- 172.21.15.16:0/439787598 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a4d87eb30 con 0x562a4d4cb070 2024-09-21T03:01:46.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e086640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15d885640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 0x562a4d878130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:01:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe15e887640 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d87b3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe150033010 tx=0x7fe150055000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcef7bfea9b023b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe143fff640 1 -- 172.21.15.16:0/439787598 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150063070 con 0x562a4d4cb070 2024-09-21T03:01:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.802+0000 7fe143fff640 1 -- 172.21.15.16:0/439787598 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe150036040 con 0x562a4d4cb070 2024-09-21T03:01:46.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.803+0000 7fe143fff640 1 -- 172.21.15.16:0/439787598 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe15002f9e0 con 0x562a4d4cb070 2024-09-21T03:01:46.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 -- 172.21.15.16:0/439787598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 msgr2=0x562a4d87b3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.806 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d87b3e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fe150033010 tx=0x7fe150055000 comp rx=0 tx=0).stop 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 -- 172.21.15.16:0/439787598 shutdown_connections 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4a21b0 0x562a4d878130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4cb070 0x562a4d87b3e0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 --2- 172.21.15.16:0/439787598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d86ea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 -- 172.21.15.16:0/439787598 >> 172.21.15.16:0/439787598 conn(0x562a4d4bf460 msgr2=0x562a4d877a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.805+0000 7fe16685e740 1 -- 172.21.15.16:0/439787598 shutdown_connections 2024-09-21T03:01:46.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.806+0000 7fe16685e740 1 -- 172.21.15.16:0/439787598 wait complete. 2024-09-21T03:01:46.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.807+0000 7fe16685e740 1 -- start start 2024-09-21T03:01:46.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.807+0000 7fe16685e740 1 Processor -- start 2024-09-21T03:01:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.808+0000 7fe16685e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d899f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.808+0000 7fe16685e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4a21b0 0x562a4d8953c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe16685e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe15d885640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4a21b0 0x562a4d8953c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe15e887640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe15e887640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33668/0 (socket says 172.21.15.16:33668) 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe15e887640 1 -- 172.21.15.16:0/1994076936 learned_addr learned my addr 172.21.15.16:0/1994076936 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe15e086640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d899f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ac090 con 0x562a4d4cb070 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ad500 con 0x562a4d4a21b0 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.809+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe1581ac760 con 0x562a4d4899f0 2024-09-21T03:01:46.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe16685e740 10 client.0 _ll_register_callbacks cb 0x7fe1480094b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:01:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe15e887640 1 -- 172.21.15.16:0/1994076936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 msgr2=0x562a4d899f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe15e887640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d899f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe15e887640 1 -- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4a21b0 msgr2=0x562a4d8953c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:01:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe15e887640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4a21b0 0x562a4d8953c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.810+0000 7fe15e887640 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a4d895e40 con 0x562a4d4cb070 2024-09-21T03:01:46.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.811+0000 7fe15e887640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe15002a900 tx=0x7fe15002f660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7023f19e4f09ae97 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:01:46.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.811+0000 7fe141ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:01:46.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe150063040 con 0x562a4d4cb070 2024-09-21T03:01:46.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fe150004180 con 0x562a4d4cb070 2024-09-21T03:01:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe150036040 con 0x562a4d4cb070 2024-09-21T03:01:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fe150004380 con 0x562a4d4cb070 2024-09-21T03:01:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe16685e740 10 client.15624 fetch_fsmap learned FSMap version 20 2024-09-21T03:01:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.812+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562a4d8fe810 con 0x562a4d4cb070 2024-09-21T03:01:46.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.813+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe15002f760 con 0x562a4d4cb070 2024-09-21T03:01:46.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.813+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fe150066070 con 0x562a4d4cb070 2024-09-21T03:01:46.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.814+0000 7fe16685e740 10 client.15624 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:01:46.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.814+0000 7fe16685e740 10 client.15624 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:01:46.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.814+0000 7fe16685e740 20 client.15624 populate_metadata read hostname 'smithi016' 2024-09-21T03:01:46.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.814+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d8febf0 0x562a4d901010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:01:46.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.815+0000 7fe16685e740 4 client.15624 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:01:46.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.815+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fe150036040 con 0x562a4d8febf0 2024-09-21T03:01:46.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.818+0000 7fe15e086640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d8febf0 0x562a4d901010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:01:46.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.820+0000 7fe141ffb640 10 client.15624 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:46.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.820+0000 7fe15e086640 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d8febf0 0x562a4d901010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a2e186767d1c9fd server_cookie=e0eeef4d49894c1e in_seq=0 out_seq=0 2024-09-21T03:01:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.821+0000 7fe141ffb640 1 -- 172.21.15.16:0/1994076936 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1422 (crc 0 0 0) 0x7fe150036040 con 0x562a4d8febf0 2024-09-21T03:01:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.821+0000 7fe141ffb640 10 client.15624 handle_command_reply: tid=0 2024-09-21T03:01:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.821+0000 7fe16685e740 1 client.15624 shutdown 2024-09-21T03:01:46.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.822+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 msgr2=0x562a4d895900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.822+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7fe15002a900 tx=0x7fe15002f660 comp rx=0 tx=0).stop 2024-09-21T03:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.822+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 shutdown_connections 2024-09-21T03:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.822+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a4d8febf0 0x562a4d901010 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.823+0000 7fe141ffb640 0 client.15624 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:01:46.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.823+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a4d4cb070 0x562a4d895900 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.823+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a4d4a21b0 0x562a4d8953c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.823+0000 7fe16685e740 1 --2- 172.21.15.16:0/1994076936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a4d4899f0 0x562a4d899f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.823+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 >> 172.21.15.16:0/1994076936 conn(0x562a4d4bf460 msgr2=0x562a4d881060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.824+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 shutdown_connections 2024-09-21T03:01:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.825+0000 7fe16685e740 1 -- 172.21.15.16:0/1994076936 wait complete. 2024-09-21T03:01:46.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.825+0000 7fe16685e740 20 client.15624 trim_cache size 0 max 16384 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "15fd7ce4-15fb-4adf-a38a-09586ce5da82", 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:46.827 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:46.828 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:01:46.829 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.826+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 msgr2=0x7fe1440b8b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.826+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 0x7fe1440b8b20 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fe150009ca0 tx=0x7fe1500040e0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.826+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 msgr2=0x7fe1581998b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.826+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fe14800e970 tx=0x7fe14800ee30 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.826+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 shutdown_connections 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe1440b6610 0x7fe1440b8b20 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe158108480 0x7fe15819eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe158107b30 0x7fe1581998b0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 --2- 172.21.15.16:0/3904253724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe1581021f0 0x7fe158199310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 >> 172.21.15.16:0/3904253724 conn(0x7fe158077c60 msgr2=0x7fe158103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 shutdown_connections 2024-09-21T03:01:46.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:01:46.827+0000 7fe1602ea640 1 -- 172.21.15.16:0/3904253724 wait complete. 2024-09-21T03:01:46.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:46 smithi123 ceph-mon[34645]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 MiB/s wr, 1.85k op/s 2024-09-21T03:01:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:46 smithi018 ceph-mon[34587]: pgmap v363: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 726 MiB/s wr, 1.85k op/s 2024-09-21T03:01:47.733 INFO:tasks.cephfs.filesystem:scrub status for tag:15fd7ce4-15fb-4adf-a38a-09586ce5da82 - {'path': '/', 'tag': '15fd7ce4-15fb-4adf-a38a-09586ce5da82', 'options': 'recursive,force'} 2024-09-21T03:01:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:48 smithi016 ceph-mon[31253]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 528 MiB/s wr, 1.15k op/s 2024-09-21T03:01:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:48 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:01:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:48 smithi018 ceph-mon[34587]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 528 MiB/s wr, 1.15k op/s 2024-09-21T03:01:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:48 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:01:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:48 smithi123 ceph-mon[34645]: pgmap v364: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 528 MiB/s wr, 1.15k op/s 2024-09-21T03:01:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:48 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:01:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:50 smithi018 ceph-mon[34587]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 585 MiB/s wr, 1.12k op/s 2024-09-21T03:01:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:50 smithi123 ceph-mon[34645]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 585 MiB/s wr, 1.12k op/s 2024-09-21T03:01:51.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:50 smithi016 ceph-mon[31253]: pgmap v365: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 585 MiB/s wr, 1.12k op/s 2024-09-21T03:01:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:52 smithi018 ceph-mon[34587]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 623 MiB/s wr, 1.13k op/s 2024-09-21T03:01:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:52 smithi123 ceph-mon[34645]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 623 MiB/s wr, 1.13k op/s 2024-09-21T03:01:53.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:52 smithi016 ceph-mon[31253]: pgmap v366: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 623 MiB/s wr, 1.13k op/s 2024-09-21T03:01:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:54 smithi018 ceph-mon[34587]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 467 MiB/s wr, 1.01k op/s 2024-09-21T03:01:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:54 smithi123 ceph-mon[34645]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 467 MiB/s wr, 1.01k op/s 2024-09-21T03:01:55.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:54 smithi016 ceph-mon[31253]: pgmap v367: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 467 MiB/s wr, 1.01k op/s 2024-09-21T03:01:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:56 smithi018 ceph-mon[34587]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 595 MiB/s wr, 1.30k op/s 2024-09-21T03:01:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:56 smithi123 ceph-mon[34645]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 595 MiB/s wr, 1.30k op/s 2024-09-21T03:01:57.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:56 smithi016 ceph-mon[31253]: pgmap v368: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.6 MiB/s rd, 595 MiB/s wr, 1.30k op/s 2024-09-21T03:01:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:01:58 smithi018 ceph-mon[34587]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 461 MiB/s wr, 699 op/s 2024-09-21T03:01:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:01:58 smithi123 ceph-mon[34645]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 461 MiB/s wr, 699 op/s 2024-09-21T03:01:59.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:01:58 smithi016 ceph-mon[31253]: pgmap v369: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 461 MiB/s wr, 699 op/s 2024-09-21T03:02:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:00 smithi018 ceph-mon[34587]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 483 MiB/s wr, 789 op/s 2024-09-21T03:02:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:00 smithi123 ceph-mon[34645]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 483 MiB/s wr, 789 op/s 2024-09-21T03:02:01.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:00 smithi016 ceph-mon[31253]: pgmap v370: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 483 MiB/s wr, 789 op/s 2024-09-21T03:02:02.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:02 smithi018 ceph-mon[34587]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 499 MiB/s wr, 855 op/s 2024-09-21T03:02:02.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:02 smithi123 ceph-mon[34645]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 499 MiB/s wr, 855 op/s 2024-09-21T03:02:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:02 smithi016 ceph-mon[31253]: pgmap v371: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 499 MiB/s wr, 855 op/s 2024-09-21T03:02:04.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:04 smithi018 ceph-mon[34587]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 394 MiB/s wr, 793 op/s 2024-09-21T03:02:04.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:04 smithi123 ceph-mon[34645]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 394 MiB/s wr, 793 op/s 2024-09-21T03:02:04.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:04 smithi016 ceph-mon[31253]: pgmap v372: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.1 MiB/s rd, 394 MiB/s wr, 793 op/s 2024-09-21T03:02:06.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:06 smithi018 ceph-mon[34587]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 504 MiB/s wr, 870 op/s 2024-09-21T03:02:06.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:06 smithi123 ceph-mon[34645]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 504 MiB/s wr, 870 op/s 2024-09-21T03:02:06.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:06 smithi016 ceph-mon[31253]: pgmap v373: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 1.2 MiB/s rd, 504 MiB/s wr, 870 op/s 2024-09-21T03:02:08.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:08 smithi018 ceph-mon[34587]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 372 MiB/s wr, 475 op/s 2024-09-21T03:02:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:08 smithi123 ceph-mon[34645]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 372 MiB/s wr, 475 op/s 2024-09-21T03:02:08.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:08 smithi016 ceph-mon[31253]: pgmap v374: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 372 MiB/s wr, 475 op/s 2024-09-21T03:02:10.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:10 smithi016 ceph-mon[31253]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 420 MiB/s wr, 430 op/s 2024-09-21T03:02:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:10 smithi018 ceph-mon[34587]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 420 MiB/s wr, 430 op/s 2024-09-21T03:02:10.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:10 smithi123 ceph-mon[34645]: pgmap v375: 97 pgs: 97 active+clean; 10 GiB data, 31 GiB used, 1.0 TiB / 1.0 TiB avail; 769 KiB/s rd, 420 MiB/s wr, 430 op/s 2024-09-21T03:02:12.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:12 smithi016 ceph-mon[31253]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 509 MiB/s wr, 488 op/s 2024-09-21T03:02:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:12 smithi018 ceph-mon[34587]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 509 MiB/s wr, 488 op/s 2024-09-21T03:02:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:12 smithi123 ceph-mon[34645]: pgmap v376: 97 pgs: 97 active+clean; 11 GiB data, 32 GiB used, 1.0 TiB / 1.0 TiB avail; 1023 KiB/s rd, 509 MiB/s wr, 488 op/s 2024-09-21T03:02:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:14 smithi018 ceph-mon[34587]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 436 MiB/s wr, 462 op/s 2024-09-21T03:02:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:14 smithi123 ceph-mon[34645]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 436 MiB/s wr, 462 op/s 2024-09-21T03:02:15.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:14 smithi016 ceph-mon[31253]: pgmap v377: 97 pgs: 97 active+clean; 11 GiB data, 33 GiB used, 1.0 TiB / 1.0 TiB avail; 652 KiB/s rd, 436 MiB/s wr, 462 op/s 2024-09-21T03:02:16.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:16 smithi018 ceph-mon[34587]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 625 MiB/s wr, 1.70k op/s 2024-09-21T03:02:16.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:16 smithi123 ceph-mon[34645]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 625 MiB/s wr, 1.70k op/s 2024-09-21T03:02:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:16 smithi016 ceph-mon[31253]: pgmap v378: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 921 KiB/s rd, 625 MiB/s wr, 1.70k op/s 2024-09-21T03:02:17.734 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:02:18.026 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:18.717 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:18 smithi016 ceph-mon[31253]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 529 MiB/s wr, 1.90k op/s 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 -- 172.21.15.16:0/935379207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 msgr2=0x7f8aa01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/935379207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa01098d0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009900 tx=0x7f8a9002a410 comp rx=0 tx=0).stop 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 -- 172.21.15.16:0/935379207 shutdown_connections 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/935379207 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa01098d0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/935379207 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 0x7f8aa0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/935379207 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 -- 172.21.15.16:0/935379207 >> 172.21.15.16:0/935379207 conn(0x7f8aa00ff610 msgr2=0x7f8aa0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 -- 172.21.15.16:0/935379207 shutdown_connections 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.715+0000 7f8aa7c40640 1 -- 172.21.15.16:0/935379207 wait complete. 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.716+0000 7f8aa7c40640 1 Processor -- start 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.716+0000 7f8aa7c40640 1 -- start start 2024-09-21T03:02:18.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa019aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 0x7f8aa019fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0113ad0 con 0x7f8aa01094c0 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0113950 con 0x7f8aa0109e10 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa7c40640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa0113c50 con 0x7f8aa0103b80 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa019aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa019aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43888/0 (socket says 172.21.15.16:43888) 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46750/0 (socket says 172.21.15.16:46750) 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa51db640 1 -- 172.21.15.16:0/2699173265 learned_addr learned my addr 172.21.15.16:0/2699173265 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 0x7f8aa019fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 -- 172.21.15.16:0/2699173265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 msgr2=0x7f8aa019fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 0x7f8aa019fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 -- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 msgr2=0x7f8aa019aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa019aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.717+0000 7f8aa59dc640 1 -- 172.21.15.16:0/2699173265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8aa01a0450 con 0x7f8aa0103b80 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c005fd0 tx=0x7f8a9c00dde0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8a96ffd640 1 -- 172.21.15.16:0/2699173265 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c009c30 con 0x7f8aa0103b80 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8a96ffd640 1 -- 172.21.15.16:0/2699173265 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c009dd0 con 0x7f8aa0103b80 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8aa01a0740 con 0x7f8aa0103b80 2024-09-21T03:02:18.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8aa01a6ea0 con 0x7f8aa0103b80 2024-09-21T03:02:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.718+0000 7f8a96ffd640 1 -- 172.21.15.16:0/2699173265 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c010460 con 0x7f8aa0103b80 2024-09-21T03:02:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.721+0000 7f8a96ffd640 1 -- 172.21.15.16:0/2699173265 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8a9c010600 con 0x7f8aa0103b80 2024-09-21T03:02:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.721+0000 7f8aae1b4740 1 Processor -- start 2024-09-21T03:02:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.721+0000 7f8aae1b4740 1 -- start start 2024-09-21T03:02:18.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a785eb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785eb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558a78561e50 con 0x558a78244070 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a78551280 con 0x558a7821b1b0 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a78555090 con 0x558a78223390 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46758/0 (socket says 172.21.15.16:46758) 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aa51db640 1 -- 172.21.15.16:0/121256862 learned_addr learned my addr 172.21.15.16:0/121256862 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a785eb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8aa51db640 1 -- 172.21.15.16:0/121256862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 msgr2=0x558a785eb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.722+0000 7f8a96ffd640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 0x7f8a8c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8a96ffd640 1 -- 172.21.15.16:0/2699173265 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8a9c0e72e0 con 0x7f8aa0103b80 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785eb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa51db640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785eb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa51db640 1 -- 172.21.15.16:0/121256862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 msgr2=0x558a785eb270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa51db640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a785eb270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa51db640 1 -- 172.21.15.16:0/121256862 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a782af650 con 0x558a78223390 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a785eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa51db640 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90031c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1dcf8099ccca5abf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 0x7f8a8c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8a8bfff640 1 -- 172.21.15.16:0/121256862 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90040070 con 0x558a78223390 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8a8bfff640 1 -- 172.21.15.16:0/121256862 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a90031e40 con 0x558a78223390 2024-09-21T03:02:18.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.723+0000 7f8a8bfff640 1 -- 172.21.15.16:0/121256862 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90035040 con 0x558a78223390 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 -- 172.21.15.16:0/121256862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 msgr2=0x558a785ebcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 0x7f8a8c0b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a98005fd0 tx=0x7f8a98004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90031c50 comp rx=0 tx=0).stop 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 -- 172.21.15.16:0/121256862 shutdown_connections 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ebcf0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785eb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/121256862 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a785eb270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 -- 172.21.15.16:0/121256862 >> 172.21.15.16:0/121256862 conn(0x558a78238460 msgr2=0x558a785e9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 -- 172.21.15.16:0/121256862 shutdown_connections 2024-09-21T03:02:18.728 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.724+0000 7f8aae1b4740 1 -- 172.21.15.16:0/121256862 wait complete. 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 -- start start 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 Processor -- start 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785f68f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 0x558a785f6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78244070 0x558a786070f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558a783b2700 con 0x558a78223390 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558a784ec110 con 0x558a78244070 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.725+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558a785fde00 con 0x558a7821b1b0 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aae1b4740 10 client.0 _ll_register_callbacks cb 0x558a78505990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 0x558a785f6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 0x558a785f6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43920/0 (socket says 172.21.15.16:43920) 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa51db640 1 -- 172.21.15.16:0/1077697652 learned_addr learned my addr 172.21.15.16:0/1077697652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785f68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78244070 msgr2=0x558a786070f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78244070 0x558a786070f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1077697652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 msgr2=0x558a785f6e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 0x558a785f6e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1077697652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a78519c40 con 0x558a7821b1b0 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785f68f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0e4a50 tx=0x7f8a9c0a9720 comp rx=0 tx=0).ready entity=mon.1 client_cookie=498a83c1a3776c77 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.726+0000 7f8a897fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c020040 con 0x558a7821b1b0 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8a9c0abed0 con 0x558a7821b1b0 2024-09-21T03:02:18.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558a78563da0 con 0x558a7821b1b0 2024-09-21T03:02:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a9c0acd10 con 0x558a7821b1b0 2024-09-21T03:02:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 10 client.25282 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:18.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a78609af0 con 0x558a7821b1b0 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558a78563da0 con 0x558a7821b1b0 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f8a9c0b60b0 con 0x558a7821b1b0 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 10 client.25282 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 10 client.25282 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 20 client.25282 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:18.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a782029f0 0x558a7860cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 4 client.25282 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.727+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558a7855f850 con 0x558a782029f0 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.728+0000 7f8aa51db640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a782029f0 0x558a7860cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.728+0000 7f8aa51db640 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a782029f0 0x558a7860cfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edc6067c90935b85 server_cookie=f70a4e73af23d912 in_seq=0 out_seq=0 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.728+0000 7f8a897fa640 10 client.25282 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8a897fa640 1 -- 172.21.15.16:0/1077697652 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558a7855f850 con 0x558a782029f0 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8a897fa640 10 client.25282 handle_command_reply: tid=0 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 client.25282 shutdown 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 msgr2=0x558a785f68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785f68f0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0e4a50 tx=0x7f8a9c0a9720 comp rx=0 tx=0).stop 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 shutdown_connections 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a782029f0 0x558a7860cfb0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78223390 0x558a785f6e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8a897fa640 0 client.25282 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78244070 0x558a786070f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1077697652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785f68f0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 >> 172.21.15.16:0/1077697652 conn(0x558a78238460 msgr2=0x558a785ee450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 shutdown_connections 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1077697652 wait complete. 2024-09-21T03:02:18.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.729+0000 7f8aae1b4740 20 client.25282 trim_cache size 0 max 16384 2024-09-21T03:02:18.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.747+0000 7f8aae1b4740 1 Processor -- start 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.747+0000 7f8aae1b4740 1 -- start start 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785ec9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78223390 0x558a78606ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01ab4e0 con 0x558a78244070 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01add60 con 0x558a78223390 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01abb50 con 0x558a7821b1b0 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa61dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa61dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43928/0 (socket says 172.21.15.16:43928) 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785ec9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa61dd640 1 -- 172.21.15.16:0/1600780724 learned_addr learned my addr 172.21.15.16:0/1600780724 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:18.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa51db640 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78223390 0x558a78606ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1600780724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78223390 msgr2=0x558a78606ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78223390 0x558a78606ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1600780724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 msgr2=0x558a78607530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa59dc640 1 -- 172.21.15.16:0/1600780724 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558a785fdd40 con 0x558a7821b1b0 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.748+0000 7f8aa61dd640 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.749+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785ec9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0ad190 tx=0x7f8a9c0036b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f11aa8b8560ef848 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.749+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/1600780724 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c020040 con 0x558a7821b1b0 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.749+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/1600780724 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c0b6b70 con 0x558a7821b1b0 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.749+0000 7f8a8b7fe640 1 -- 172.21.15.16:0/1600780724 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a9c0982e0 con 0x558a7821b1b0 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1600780724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 msgr2=0x558a785ec9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785ec9b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c0ad190 tx=0x7f8a9c0036b0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1600780724 shutdown_connections 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78607530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a78223390 0x558a78606ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/1600780724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a7821b1b0 0x558a785ec9b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1600780724 >> 172.21.15.16:0/1600780724 conn(0x558a78238460 msgr2=0x558a785ec2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1600780724 shutdown_connections 2024-09-21T03:02:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- 172.21.15.16:0/1600780724 wait complete. 2024-09-21T03:02:18.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.750+0000 7f8aae1b4740 1 -- start start 2024-09-21T03:02:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 Processor -- start 2024-09-21T03:02:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78676970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01ac220 con 0x558a78244070 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01ad690 con 0x558a7821b1b0 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8aa01ac8f0 con 0x558a78223390 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aae1b4740 10 client.0 _ll_register_callbacks cb 0x7f8a9c0037e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa61dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78676970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa59dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa51db640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:46772/0 (socket says 172.21.15.16:46772) 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa59dc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44716/0 (socket says 172.21.15.16:44716) 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa61dd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78676970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43938/0 (socket says 172.21.15.16:43938) 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.751+0000 7f8aa51db640 1 -- 172.21.15.16:0/312464495 learned_addr learned my addr 172.21.15.16:0/312464495 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 -- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 msgr2=0x558a785ff8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 -- 172.21.15.16:0/312464495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 msgr2=0x558a78676970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78676970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 -- 172.21.15.16:0/312464495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558a78676fe0 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aa51db640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90005210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=10f05c2164a01936 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8a897fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a90040030 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8a90004240 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.752+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a9c0b6b70 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8a90035040 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 10 client.25288 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558a786782b0 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a9c0b6b70 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f8a90004aa0 con 0x558a78223390 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 10 client.25288 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:18.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 10 client.25288 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 20 client.25288 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a781d7da0 0x558a7867a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 4 client.25288 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558a7855cc80 con 0x558a781d7da0 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.753+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a781d7da0 0x558a7867a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8a897fa640 10 client.25288 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:18.758 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8aa59dc640 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a781d7da0 0x558a7867a650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1c45eb37f397dda server_cookie=9538514473dfefe8 in_seq=0 out_seq=0 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8a897fa640 1 -- 172.21.15.16:0/312464495 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1226 (crc 0 0 0) 0x558a7855cc80 con 0x558a781d7da0 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8a897fa640 10 client.25288 handle_command_reply: tid=0 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8aae1b4740 1 client.25288 shutdown 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 msgr2=0x558a785ffe80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.754+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8a90009230 tx=0x7f8a90005210 comp rx=0 tx=0).stop 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 shutdown_connections 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558a781d7da0 0x558a7867a650 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558a78244070 0x558a78676970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8a897fa640 0 client.25288 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558a78223390 0x558a785ffe80 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 --2- 172.21.15.16:0/312464495 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558a7821b1b0 0x558a785ff8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 >> 172.21.15.16:0/312464495 conn(0x558a78238460 msgr2=0x558a785f8710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 shutdown_connections 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 1 -- 172.21.15.16:0/312464495 wait complete. 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.755+0000 7f8aae1b4740 20 client.25288 trim_cache size 0 max 16384 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:02:18.760 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:18.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:18.762 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 msgr2=0x7f8a8c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 0x7f8a8c0b8b90 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f8a98005fd0 tx=0x7f8a98004a40 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 msgr2=0x7f8aa019a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f8a9c005fd0 tx=0x7f8a9c00dde0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 shutdown_connections 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8a8c0b6680 0x7f8a8c0b8b90 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8aa01094c0 0x7f8aa019aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8aa0109e10 0x7f8aa019fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 --2- 172.21.15.16:0/2699173265 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8aa0103b80 0x7f8aa019a4e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 >> 172.21.15.16:0/2699173265 conn(0x7f8aa00ff610 msgr2=0x7f8aa0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 shutdown_connections 2024-09-21T03:02:18.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:18.756+0000 7f8aa7c40640 1 -- 172.21.15.16:0/2699173265 wait complete. 2024-09-21T03:02:18.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:18 smithi018 ceph-mon[34587]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 529 MiB/s wr, 1.90k op/s 2024-09-21T03:02:18.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:18 smithi123 ceph-mon[34645]: pgmap v379: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 529 MiB/s wr, 1.90k op/s 2024-09-21T03:02:19.577 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:02:19.577 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:02:20.346 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:20.812 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:20 smithi016 ceph-mon[31253]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 552 MiB/s wr, 2.33k op/s 2024-09-21T03:02:20.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:20 smithi018 ceph-mon[34587]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 552 MiB/s wr, 2.33k op/s 2024-09-21T03:02:20.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:20 smithi123 ceph-mon[34645]: pgmap v380: 97 pgs: 97 active+clean; 13 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 552 MiB/s wr, 2.33k op/s 2024-09-21T03:02:21.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 -- 172.21.15.16:0/2294930937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac075dd0 msgr2=0x7fdaac0761e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 --2- 172.21.15.16:0/2294930937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac075dd0 0x7fdaac0761e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0009900 tx=0x7fdaa002a410 comp rx=0 tx=0).stop 2024-09-21T03:02:21.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 -- 172.21.15.16:0/2294930937 shutdown_connections 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 --2- 172.21.15.16:0/2294930937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 0x7fdaac116930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 --2- 172.21.15.16:0/2294930937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac075dd0 0x7fdaac0761e0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 --2- 172.21.15.16:0/2294930937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac077380 0x7fdaac0757b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.301+0000 7fdab310f640 1 -- 172.21.15.16:0/2294930937 >> 172.21.15.16:0/2294930937 conn(0x7fdaac0ff5e0 msgr2=0x7fdaac101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.302+0000 7fdab310f640 1 -- 172.21.15.16:0/2294930937 shutdown_connections 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.302+0000 7fdab310f640 1 -- 172.21.15.16:0/2294930937 wait complete. 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.304+0000 7fdab310f640 1 Processor -- start 2024-09-21T03:02:21.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.304+0000 7fdab310f640 1 -- start start 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdab310f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac075dd0 0x7fdaac1a0a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdab310f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdab310f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 0x7fdaac1a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdaabfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdaabfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44746/0 (socket says 172.21.15.16:44746) 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.305+0000 7fdaabfff640 1 -- 172.21.15.16:0/3293386441 learned_addr learned my addr 172.21.15.16:0/3293386441 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:21.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdab16ac640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 0x7fdaac1a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac11a070 con 0x7fdaac075dd0 2024-09-21T03:02:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac119ef0 con 0x7fdaac077380 2024-09-21T03:02:21.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fdaac11a1f0 con 0x7fdaac114460 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdab0eab640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac075dd0 0x7fdaac1a0a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdaabfff640 1 -- 172.21.15.16:0/3293386441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 msgr2=0x7fdaac1a6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdaabfff640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 0x7fdaac1a6280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdaabfff640 1 -- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac075dd0 msgr2=0x7fdaac1a0a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdaabfff640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac075dd0 0x7fdaac1a0a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.306+0000 7fdaabfff640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdaac1a69b0 con 0x7fdaac077380 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.307+0000 7fdaabfff640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0032fd0 tx=0x7fdaa0031a20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.307+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa003f030 con 0x7fdaac077380 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.307+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdaa00043d0 con 0x7fdaac077380 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.307+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdaa0035070 con 0x7fdaac077380 2024-09-21T03:02:21.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.307+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdaac1a6c40 con 0x7fdaac077380 2024-09-21T03:02:21.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.308+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdaac1a7010 con 0x7fdaac077380 2024-09-21T03:02:21.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.312+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fdaa0042030 con 0x7fdaac077380 2024-09-21T03:02:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.313+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdaac0761e0 con 0x7fdaac077380 2024-09-21T03:02:21.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.314+0000 7fdaa9ffb640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 0x7fda940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.314+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fdaa0107aa0 con 0x7fdaac077380 2024-09-21T03:02:21.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.315+0000 7fdab0eab640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 0x7fda940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:21.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.316+0000 7fdab0eab640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 0x7fda940b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009b60 tx=0x7fda9c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:21.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.320+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fdaa00ce010 con 0x7fdaac077380 2024-09-21T03:02:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.775+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fdaac112c00 con 0x7fdaac077380 2024-09-21T03:02:21.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.776+0000 7fdaa9ffb640 1 -- 172.21.15.16:0/3293386441 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7fdaa00d23f0 con 0x7fdaac077380 2024-09-21T03:02:21.778 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:02:21.778 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:02:21.778 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 msgr2=0x7fda940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 0x7fda940b8c10 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fda9c009b60 tx=0x7fda9c005f90 comp rx=0 tx=0).stop 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 msgr2=0x7fdaac1a0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fdaa0032fd0 tx=0x7fdaa0031a20 comp rx=0 tx=0).stop 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 shutdown_connections 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fda940b6700 0x7fda940b8c10 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdaac114460 0x7fdaac1a6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fdaac077380 0x7fdaac1a0fe0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 --2- 172.21.15.16:0/3293386441 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fdaac075dd0 0x7fdaac1a0a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.779+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 >> 172.21.15.16:0/3293386441 conn(0x7fdaac0ff5e0 msgr2=0x7fdaac110360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.780+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 shutdown_connections 2024-09-21T03:02:21.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:21.780+0000 7fdab310f640 1 -- 172.21.15.16:0/3293386441 wait complete. 2024-09-21T03:02:22.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:22 smithi016 ceph-mon[31253]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 544 MiB/s wr, 2.81k op/s 2024-09-21T03:02:22.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:22 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3293386441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:22.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:22 smithi018 ceph-mon[34587]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 544 MiB/s wr, 2.81k op/s 2024-09-21T03:02:22.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:22 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3293386441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:22.958 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:02:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:22 smithi123 ceph-mon[34645]: pgmap v381: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 561 KiB/s rd, 544 MiB/s wr, 2.81k op/s 2024-09-21T03:02:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:22 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3293386441' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:02:23.436 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:24.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 -- 172.21.15.16:0/67690504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c108d20 msgr2=0x7f083c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 --2- 172.21.15.16:0/67690504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c108d20 0x7f083c10f550 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7f0834009ca0 tx=0x7f083402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 -- 172.21.15.16:0/67690504 shutdown_connections 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 --2- 172.21.15.16:0/67690504 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c108d20 0x7f083c10f550 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 --2- 172.21.15.16:0/67690504 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c1083d0 0x7f083c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 --2- 172.21.15.16:0/67690504 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c102ae0 0x7f083c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.164+0000 7f0840986640 1 -- 172.21.15.16:0/67690504 >> 172.21.15.16:0/67690504 conn(0x7f083c0797b0 msgr2=0x7f083c079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.168+0000 7f0840986640 1 -- 172.21.15.16:0/67690504 shutdown_connections 2024-09-21T03:02:24.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.168+0000 7f0840986640 1 -- 172.21.15.16:0/67690504 wait complete. 2024-09-21T03:02:24.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.170+0000 7f0840986640 1 Processor -- start 2024-09-21T03:02:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.170+0000 7f0840986640 1 -- start start 2024-09-21T03:02:24.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.171+0000 7f0840986640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.171+0000 7f0840986640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c1083d0 0x7f083c10f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f0840986640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c108d20 0x7f083c1a2f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f0840986640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c113b30 con 0x7f083c102ae0 2024-09-21T03:02:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f0840986640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1139b0 con 0x7f083c1083d0 2024-09-21T03:02:24.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f0840986640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f083c113cb0 con 0x7f083c108d20 2024-09-21T03:02:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f083a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f083a59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43982/0 (socket says 172.21.15.16:43982) 2024-09-21T03:02:24.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.172+0000 7f083a59c640 1 -- 172.21.15.16:0/2472357572 learned_addr learned my addr 172.21.15.16:0/2472357572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:24.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.173+0000 7f083a59c640 1 -- 172.21.15.16:0/2472357572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c108d20 msgr2=0x7f083c1a2f50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:02:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.173+0000 7f083a59c640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c108d20 0x7f083c1a2f50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.173+0000 7f083a59c640 1 -- 172.21.15.16:0/2472357572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c1083d0 msgr2=0x7f083c10f0c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:02:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.173+0000 7f083a59c640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c1083d0 0x7f083c10f0c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.173+0000 7f083a59c640 1 -- 172.21.15.16:0/2472357572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1a3680 con 0x7f083c102ae0 2024-09-21T03:02:24.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.174+0000 7f083a59c640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d950 tx=0x7f082c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.174+0000 7f082b7fe640 1 -- 172.21.15.16:0/2472357572 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c017070 con 0x7f083c102ae0 2024-09-21T03:02:24.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.174+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c1a3970 con 0x7f083c102ae0 2024-09-21T03:02:24.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.175+0000 7f082b7fe640 1 -- 172.21.15.16:0/2472357572 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082c009bb0 con 0x7f083c102ae0 2024-09-21T03:02:24.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.175+0000 7f082b7fe640 1 -- 172.21.15.16:0/2472357572 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c020060 con 0x7f083c102ae0 2024-09-21T03:02:24.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.176+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c1ad310 con 0x7f083c102ae0 2024-09-21T03:02:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.182+0000 7f0846efa740 1 Processor -- start 2024-09-21T03:02:24.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.183+0000 7f0846efa740 1 -- start start 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04bbe070 0x55ce04f656a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b951b0 0x55ce04f65be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce04edbea0 con 0x55ce04b951b0 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ce04ecb2d0 con 0x55ce04b9d390 2024-09-21T03:02:24.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.184+0000 7f0846efa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ce04ecf0e0 con 0x55ce04bbe070 2024-09-21T03:02:24.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.185+0000 7f0839d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.185+0000 7f0839d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44770/0 (socket says 172.21.15.16:44770) 2024-09-21T03:02:24.188 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.185+0000 7f0839d9b640 1 -- 172.21.15.16:0/3205462795 learned_addr learned my addr 172.21.15.16:0/3205462795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.187+0000 7f0839d9b640 1 -- 172.21.15.16:0/3205462795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04bbe070 msgr2=0x55ce04f656a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.187+0000 7f0839d9b640 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04bbe070 0x55ce04f656a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.187+0000 7f0839d9b640 1 -- 172.21.15.16:0/3205462795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b951b0 msgr2=0x55ce04f65be0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.187+0000 7f0839d9b640 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b951b0 0x55ce04f65be0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.188+0000 7f0839d9b640 1 -- 172.21.15.16:0/3205462795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce04c29650 con 0x55ce04b9d390 2024-09-21T03:02:24.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.188+0000 7f082b7fe640 1 -- 172.21.15.16:0/2472357572 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f082c010460 con 0x7f083c102ae0 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.194+0000 7f082b7fe640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 0x7f08240b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.195+0000 7f0839d9b640 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08300096e0 tx=0x7f0830009ba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=622ca0c1b72b2961 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.195+0000 7f082b7fe640 1 -- 172.21.15.16:0/2472357572 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f082c0e7080 con 0x7f083c102ae0 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.196+0000 7f0828ff9640 1 -- 172.21.15.16:0/3205462795 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830015020 con 0x55ce04b9d390 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.196+0000 7f0828ff9640 1 -- 172.21.15.16:0/3205462795 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083000bd70 con 0x55ce04b9d390 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.196+0000 7f0828ff9640 1 -- 172.21.15.16:0/3205462795 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830010380 con 0x55ce04b9d390 2024-09-21T03:02:24.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.196+0000 7f083ad9d640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 0x7f08240b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.197+0000 7f0846efa740 1 -- 172.21.15.16:0/3205462795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 msgr2=0x55ce04f66120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.197+0000 7f0846efa740 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f08300096e0 tx=0x7f0830009ba0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f083ad9d640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 0x7f08240b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f083402aed0 tx=0x7f0834032fe0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f0846efa740 1 -- 172.21.15.16:0/3205462795 shutdown_connections 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f0846efa740 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b951b0 0x55ce04f65be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f0846efa740 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f66120 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f0846efa740 1 --2- 172.21.15.16:0/3205462795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04bbe070 0x55ce04f656a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.198+0000 7f0846efa740 1 -- 172.21.15.16:0/3205462795 >> 172.21.15.16:0/3205462795 conn(0x55ce04bb2460 msgr2=0x55ce04f63590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.202+0000 7f0846efa740 1 -- 172.21.15.16:0/3205462795 shutdown_connections 2024-09-21T03:02:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.202+0000 7f0846efa740 1 -- 172.21.15.16:0/3205462795 wait complete. 2024-09-21T03:02:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.203+0000 7f0846efa740 1 -- start start 2024-09-21T03:02:24.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.203+0000 7f0846efa740 1 Processor -- start 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f70de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f71320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ce04d2c700 con 0x55ce04bbe070 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ce04e66160 con 0x55ce04b9d390 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ce04f73e80 con 0x55ce04b951b0 2024-09-21T03:02:24.209 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.204+0000 7f0846efa740 10 client.0 _ll_register_callbacks cb 0x55ce04e7f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.206+0000 7f083ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.206+0000 7f083ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43984/0 (socket says 172.21.15.16:43984) 2024-09-21T03:02:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.206+0000 7f083ad9d640 1 -- 172.21.15.16:0/3674193107 learned_addr learned my addr 172.21.15.16:0/3674193107 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:24.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083a59c640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f70de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f0839d9b640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f71320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083ad9d640 1 -- 172.21.15.16:0/3674193107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 msgr2=0x55ce04f70de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083ad9d640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f70de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083ad9d640 1 -- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 msgr2=0x55ce04f71320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083ad9d640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f71320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.207+0000 7f083ad9d640 1 -- 172.21.15.16:0/3674193107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce04e93c90 con 0x55ce04bbe070 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.208+0000 7f083ad9d640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0834031c50 tx=0x7f083402a8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a44d1dfff4a761b9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.208+0000 7f0821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:02:24.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.210+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083404c070 con 0x55ce04bbe070 2024-09-21T03:02:24.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.210+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f08340047c0 con 0x55ce04bbe070 2024-09-21T03:02:24.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.212+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f083000bd70 con 0x55ce04bbe070 2024-09-21T03:02:24.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.214+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0834056050 con 0x55ce04bbe070 2024-09-21T03:02:24.215 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.214+0000 7f0846efa740 10 client.15657 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:24.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.214+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ce04f7fbf0 con 0x55ce04bbe070 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.215+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f083000bd70 con 0x55ce04bbe070 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.215+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f08340485d0 con 0x55ce04bbe070 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.215+0000 7f0846efa740 10 client.15657 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.215+0000 7f0846efa740 10 client.15657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.215+0000 7f0846efa740 20 client.15657 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.216+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b7c9f0 0x55ce04f830e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.216+0000 7f0846efa740 4 client.15657 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.216+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ce04edddf0 con 0x55ce04b7c9f0 2024-09-21T03:02:24.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.216+0000 7f083a59c640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b7c9f0 0x55ce04f830e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.218+0000 7f083a59c640 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b7c9f0 0x55ce04f830e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6cdc8bef535db9a4 server_cookie=1e91a302dc8408e0 in_seq=0 out_seq=0 2024-09-21T03:02:24.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.219+0000 7f0821ffb640 10 client.15657 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.221+0000 7f0821ffb640 1 -- 172.21.15.16:0/3674193107 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ce04edddf0 con 0x55ce04b7c9f0 2024-09-21T03:02:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.222+0000 7f0821ffb640 10 client.15657 handle_command_reply: tid=0 2024-09-21T03:02:24.224 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.222+0000 7f0846efa740 1 client.15657 shutdown 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.235+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 msgr2=0x55ce04f7d1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.235+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f0834031c50 tx=0x7f083402a8e0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 shutdown_connections 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b7c9f0 0x55ce04f830e0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f7d1a0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b9d390 0x55ce04f71320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 --2- 172.21.15.16:0/3674193107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f70de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 >> 172.21.15.16:0/3674193107 conn(0x55ce04bb2460 msgr2=0x55ce04f68830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.236+0000 7f0821ffb640 0 client.15657 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.237+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 shutdown_connections 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.237+0000 7f0846efa740 1 -- 172.21.15.16:0/3674193107 wait complete. 2024-09-21T03:02:24.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.238+0000 7f0846efa740 20 client.15657 trim_cache size 0 max 16384 2024-09-21T03:02:24.277 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.275+0000 7f0846efa740 1 Processor -- start 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.275+0000 7f0846efa740 1 -- start start 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.276+0000 7f0846efa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f712a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.277+0000 7f0846efa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.277+0000 7f0846efa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04bbe070 0x55ce04f63100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.278+0000 7f0839d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.278+0000 7f0839d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43996/0 (socket says 172.21.15.16:43996) 2024-09-21T03:02:24.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.279+0000 7f0839d9b640 1 -- 172.21.15.16:0/4006027610 learned_addr learned my addr 172.21.15.16:0/4006027610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.279+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b3e10 con 0x55ce04b9d390 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.279+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b6690 con 0x55ce04bbe070 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.279+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b4480 con 0x55ce04b951b0 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.280+0000 7f083ad9d640 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04bbe070 0x55ce04f63100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.280+0000 7f083a59c640 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f712a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.281+0000 7f0839d9b640 1 -- 172.21.15.16:0/4006027610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 msgr2=0x55ce04f712a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.281+0000 7f0839d9b640 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f712a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.281+0000 7f0839d9b640 1 -- 172.21.15.16:0/4006027610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04bbe070 msgr2=0x55ce04f63100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.281+0000 7f0839d9b640 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04bbe070 0x55ce04f63100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.281+0000 7f0839d9b640 1 -- 172.21.15.16:0/4006027610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ce04f720a0 con 0x55ce04b9d390 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.283+0000 7f0839d9b640 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0830006f20 tx=0x7f0830009430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63e80dfaa03ce8a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.285+0000 7f0823fff640 1 -- 172.21.15.16:0/4006027610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0830015040 con 0x55ce04b9d390 2024-09-21T03:02:24.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.285+0000 7f0823fff640 1 -- 172.21.15.16:0/4006027610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0830004040 con 0x55ce04b9d390 2024-09-21T03:02:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.288+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 msgr2=0x55ce04f62bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.288+0000 7f0846efa740 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f0830006f20 tx=0x7f0830009430 comp rx=0 tx=0).stop 2024-09-21T03:02:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.292+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 shutdown_connections 2024-09-21T03:02:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.292+0000 7f0846efa740 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04b9d390 0x55ce04f62bc0 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.292+0000 7f0846efa740 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04bbe070 0x55ce04f63100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.292+0000 7f0846efa740 1 --2- 172.21.15.16:0/4006027610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b951b0 0x55ce04f712a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.296 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.292+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 >> 172.21.15.16:0/4006027610 conn(0x55ce04bb2460 msgr2=0x55ce04f70b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.294+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 shutdown_connections 2024-09-21T03:02:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.295+0000 7f0846efa740 1 -- 172.21.15.16:0/4006027610 wait complete. 2024-09-21T03:02:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.296+0000 7f0846efa740 1 -- start start 2024-09-21T03:02:24.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.296+0000 7f0846efa740 1 Processor -- start 2024-09-21T03:02:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.299+0000 7f0846efa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b951b0 0x55ce04f751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.306+0000 7f0846efa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b9d390 0x55ce04f75720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.306+0000 7f0846efa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.307+0000 7f0846efa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b4b50 con 0x55ce04bbe070 2024-09-21T03:02:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.307+0000 7f0846efa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b5fc0 con 0x55ce04b951b0 2024-09-21T03:02:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.307+0000 7f0846efa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f083c1b5220 con 0x55ce04b9d390 2024-09-21T03:02:24.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.308+0000 7f0846efa740 10 client.0 _ll_register_callbacks cb 0x7f083403f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.311+0000 7f083ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.311+0000 7f083ad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34462/0 (socket says 172.21.15.16:34462) 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.311+0000 7f083ad9d640 1 -- 172.21.15.16:0/2000179880 learned_addr learned my addr 172.21.15.16:0/2000179880 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.312+0000 7f083a59c640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b951b0 0x55ce04f751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.312+0000 7f0839d9b640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b9d390 0x55ce04f75720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.315+0000 7f083ad9d640 1 -- 172.21.15.16:0/2000179880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b9d390 msgr2=0x55ce04f75720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.315+0000 7f083ad9d640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b9d390 0x55ce04f75720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.315+0000 7f083ad9d640 1 -- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b951b0 msgr2=0x55ce04f751e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:24.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.315+0000 7f083ad9d640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b951b0 0x55ce04f751e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.315+0000 7f083ad9d640 1 -- 172.21.15.16:0/2000179880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ce04fecba0 con 0x55ce04bbe070 2024-09-21T03:02:24.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.316+0000 7f083ad9d640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0834048970 tx=0x7f0834001190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bafbae9ed7430e1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:24.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.319+0000 7f0821ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:02:24.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.319+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f083404c040 con 0x55ce04bbe070 2024-09-21T03:02:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.320+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ce04ed98a0 con 0x55ce04bbe070 2024-09-21T03:02:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.320+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f08340044c0 con 0x55ce04bbe070 2024-09-21T03:02:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.321+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f08340046c0 con 0x55ce04bbe070 2024-09-21T03:02:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.322+0000 7f0846efa740 10 client.15663 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:24.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.322+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ce04fedeb0 con 0x55ce04bbe070 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.325+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ce04ed98a0 con 0x55ce04bbe070 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.325+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f083406c020 con 0x55ce04bbe070 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.328+0000 7f0846efa740 10 client.15663 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.329+0000 7f0846efa740 10 client.15663 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.329+0000 7f0846efa740 20 client.15663 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.329+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b51da0 0x55ce04ff0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.329+0000 7f0846efa740 4 client.15663 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:02:24.333 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.329+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ce04ed6cd0 con 0x55ce04b51da0 2024-09-21T03:02:24.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.331+0000 7f083a59c640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b51da0 0x55ce04ff0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.336+0000 7f083a59c640 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b51da0 0x55ce04ff0560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=891f5530aba8c9b7 server_cookie=31931fb79dc2c7b1 in_seq=0 out_seq=0 2024-09-21T03:02:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.337+0000 7f0821ffb640 10 client.15663 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.337+0000 7f0821ffb640 1 -- 172.21.15.16:0/2000179880 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ce04ed6cd0 con 0x55ce04b51da0 2024-09-21T03:02:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.337+0000 7f0821ffb640 10 client.15663 handle_command_reply: tid=0 2024-09-21T03:02:24.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.337+0000 7f0846efa740 1 client.15663 shutdown 2024-09-21T03:02:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 msgr2=0x55ce04f75c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f0834048970 tx=0x7f0834001190 comp rx=0 tx=0).stop 2024-09-21T03:02:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 shutdown_connections 2024-09-21T03:02:24.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ce04b51da0 0x55ce04ff0560 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ce04bbe070 0x55ce04f75c60 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ce04b9d390 0x55ce04f75720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.341+0000 7f0846efa740 1 --2- 172.21.15.16:0/2000179880 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ce04b951b0 0x55ce04f751e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.342+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 >> 172.21.15.16:0/2000179880 conn(0x55ce04bb2460 msgr2=0x55ce04f6f220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.342+0000 7f0821ffb640 0 client.15663 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.346+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 shutdown_connections 2024-09-21T03:02:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.346+0000 7f0846efa740 1 -- 172.21.15.16:0/2000179880 wait complete. 2024-09-21T03:02:24.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.346+0000 7f0846efa740 20 client.15663 trim_cache size 0 max 16384 2024-09-21T03:02:24.352 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:02:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.351+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 msgr2=0x7f08240b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.351+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 0x7f08240b8c40 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f083402aed0 tx=0x7f0834032fe0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.351+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 msgr2=0x7f083c10eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.351+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f082c00d950 tx=0x7f082c00de10 comp rx=0 tx=0).stop 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 shutdown_connections 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b6730 0x7f08240b8c40 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c108d20 0x7f083c1a2f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c1083d0 0x7f083c10f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 --2- 172.21.15.16:0/2472357572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c102ae0 0x7f083c10eb20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.352+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 >> 172.21.15.16:0/2472357572 conn(0x7f083c0797b0 msgr2=0x7f083c1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.353+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 shutdown_connections 2024-09-21T03:02:24.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:24.353+0000 7f0840986640 1 -- 172.21.15.16:0/2472357572 wait complete. 2024-09-21T03:02:25.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:24 smithi016 ceph-mon[31253]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 417 MiB/s wr, 2.62k op/s 2024-09-21T03:02:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:24 smithi018 ceph-mon[34587]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 417 MiB/s wr, 2.62k op/s 2024-09-21T03:02:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:24 smithi123 ceph-mon[34645]: pgmap v382: 97 pgs: 97 active+clean; 13 GiB data, 40 GiB used, 1.0 TiB / 1.0 TiB avail; 610 KiB/s rd, 417 MiB/s wr, 2.62k op/s 2024-09-21T03:02:26.235 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:02:26.235 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:02:26.644 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:27.073 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:26 smithi016 ceph-mon[31253]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 565 MiB/s wr, 2.69k op/s 2024-09-21T03:02:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:26 smithi018 ceph-mon[34587]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 565 MiB/s wr, 2.69k op/s 2024-09-21T03:02:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:26 smithi123 ceph-mon[34645]: pgmap v383: 97 pgs: 97 active+clean; 14 GiB data, 41 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 565 MiB/s wr, 2.69k op/s 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.292+0000 7f4dc6ffd640 1 -- 172.21.15.16:0/4131257304 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc030850 con 0x7f4dc8111f40 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.293+0000 7f4dce900640 1 -- 172.21.15.16:0/4131257304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc8111f40 msgr2=0x7f4dc810a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.293+0000 7f4dce900640 1 --2- 172.21.15.16:0/4131257304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc8111f40 0x7f4dc810a3c0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc009860 tx=0x7f4dbc02a340 comp rx=0 tx=0).stop 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 -- 172.21.15.16:0/4131257304 shutdown_connections 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 --2- 172.21.15.16:0/4131257304 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc811cc30 0x7f4dc811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 --2- 172.21.15.16:0/4131257304 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc810adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 --2- 172.21.15.16:0/4131257304 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc8111f40 0x7f4dc810a3c0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 -- 172.21.15.16:0/4131257304 >> 172.21.15.16:0/4131257304 conn(0x7f4dc8070ec0 msgr2=0x7f4dc80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.297 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.294+0000 7f4dce900640 1 -- 172.21.15.16:0/4131257304 shutdown_connections 2024-09-21T03:02:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.295+0000 7f4dce900640 1 -- 172.21.15.16:0/4131257304 wait complete. 2024-09-21T03:02:27.298 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.296+0000 7f4dce900640 1 Processor -- start 2024-09-21T03:02:27.299 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.296+0000 7f4dce900640 1 -- start start 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.297+0000 7f4dce900640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.297+0000 7f4dce900640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 0x7f4dc8115680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.297+0000 7f4dce900640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 0x7f4dc8115bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.297+0000 7f4dce900640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8122900 con 0x7f4dc8111f40 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.298+0000 7f4dce900640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8122780 con 0x7f4dc810a9e0 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.298+0000 7f4dce900640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc8122a80 con 0x7f4dc811cc30 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.299+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.299+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 0x7f4dc8115680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.300 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.299+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53264/0 (socket says 172.21.15.16:53264) 2024-09-21T03:02:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.299+0000 7f4dc7fff640 1 -- 172.21.15.16:0/460982609 learned_addr learned my addr 172.21.15.16:0/460982609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.299+0000 7f4dcce9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 0x7f4dc8115bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 -- 172.21.15.16:0/460982609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 msgr2=0x7f4dc8115bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 0x7f4dc8115bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 -- 172.21.15.16:0/460982609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 msgr2=0x7f4dc811c3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 -- 172.21.15.16:0/460982609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4dc81163c0 con 0x7f4dc8111f40 2024-09-21T03:02:27.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.300+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 0x7f4dc8115680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806c6e0 tx=0x7f4db800cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dc57fa640 1 -- 172.21.15.16:0/460982609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db8004220 con 0x7f4dc8111f40 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 0x7f4dc8115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4dc811b080 con 0x7f4dc8111f40 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dc57fa640 1 -- 172.21.15.16:0/460982609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db80043c0 con 0x7f4dc8111f40 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dc57fa640 1 -- 172.21.15.16:0/460982609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db800f450 con 0x7f4dc8111f40 2024-09-21T03:02:27.303 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.301+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4dc811b610 con 0x7f4dc8111f40 2024-09-21T03:02:27.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.303+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.308+0000 7f4dd4e74740 1 Processor -- start 2024-09-21T03:02:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.308+0000 7f4dc57fa640 1 -- 172.21.15.16:0/460982609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4db80079b0 con 0x7f4dc8111f40 2024-09-21T03:02:27.310 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.309+0000 7f4dd4e74740 1 -- start start 2024-09-21T03:02:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.309+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.310+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052cd990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.310+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052cded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.310+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a05243e60 con 0x562a04f26350 2024-09-21T03:02:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.310+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562a05233290 con 0x562a04f05670 2024-09-21T03:02:27.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.310+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562a052370a0 con 0x562a04efd490 2024-09-21T03:02:27.313 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.311+0000 7f4dcce9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.311+0000 7f4dcce9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34484/0 (socket says 172.21.15.16:34484) 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.311+0000 7f4dcce9d640 1 -- 172.21.15.16:0/3818301271 learned_addr learned my addr 172.21.15.16:0/3818301271 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.311+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052cded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.311+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dc57fa640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 0x7f4db00b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 -- 172.21.15.16:0/3818301271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 msgr2=0x562a052cd990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052cd990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 -- 172.21.15.16:0/3818301271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 msgr2=0x562a052cded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052cded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 -- 172.21.15.16:0/3818301271 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a04f91930 con 0x562a04f26350 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dc57fa640 1 -- 172.21.15.16:0/460982609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4db80e72e0 con 0x7f4dc8111f40 2024-09-21T03:02:27.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:02:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.312+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000f720 tx=0x7f4dc000fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab6dd8d0c3b61acd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.313+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 0x7f4db00b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.313+0000 7f4dae7fc640 1 -- 172.21.15.16:0/3818301271 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc001e070 con 0x562a04f26350 2024-09-21T03:02:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.313+0000 7f4dae7fc640 1 -- 172.21.15.16:0/3818301271 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4dc0018410 con 0x562a04f26350 2024-09-21T03:02:27.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.313+0000 7f4dae7fc640 1 -- 172.21.15.16:0/3818301271 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dc000c6d0 con 0x562a04f26350 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.314+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3818301271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 msgr2=0x562a052cd450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.314+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f4dc000f720 tx=0x7f4dc000fbe0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3818301271 shutdown_connections 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052cded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052cd990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3818301271 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f26350 0x562a052cd450 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 0x7f4db00b8c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001b000 tx=0x7f4dc000c870 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3818301271 >> 172.21.15.16:0/3818301271 conn(0x562a04f1a740 msgr2=0x562a052cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3818301271 shutdown_connections 2024-09-21T03:02:27.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.315+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3818301271 wait complete. 2024-09-21T03:02:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.316+0000 7f4dd4e74740 1 -- start start 2024-09-21T03:02:27.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.317+0000 7f4dd4e74740 1 Processor -- start 2024-09-21T03:02:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.317+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.317+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052d94f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47672/0 (socket says 172.21.15.16:47672) 2024-09-21T03:02:27.319 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 -- 172.21.15.16:0/3340639865 learned_addr learned my addr 172.21.15.16:0/3340639865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f26350 0x562a052e4f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562a05094890 con 0x562a04f05670 2024-09-21T03:02:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562a051cdd40 con 0x562a04f26350 2024-09-21T03:02:27.320 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562a052dbc30 con 0x562a04efd490 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dd4e74740 10 client.0 _ll_register_callbacks cb 0x562a05231c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 -- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f26350 msgr2=0x562a052e4f20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f26350 0x562a052e4f20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 -- 172.21.15.16:0/3340639865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 msgr2=0x562a052d94f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052d94f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.318+0000 7f4dc7fff640 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a052e5490 con 0x562a04efd490 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.319+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4dc0018410 con 0x562a04efd490 2024-09-21T03:02:27.321 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.319+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0303f0 tx=0x7f4dbc004060 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a93e9b9c75bccb88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.319+0000 7f4da77fe640 10 client.25309 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.320+0000 7f4da77fe640 1 -- 172.21.15.16:0/3340639865 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc037040 con 0x562a04efd490 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.320+0000 7f4da77fe640 1 -- 172.21.15.16:0/3340639865 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4dbc030650 con 0x562a04efd490 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.320+0000 7f4da77fe640 1 -- 172.21.15.16:0/3340639865 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4dbc030850 con 0x562a04efd490 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.320+0000 7f4dd4e74740 10 client.25309 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:27.322 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.320+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=20}) -- 0x562a052e7c80 con 0x562a04efd490 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.322+0000 7f4da77fe640 1 -- 172.21.15.16:0/3340639865 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4dbc050060 con 0x562a04efd490 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.322+0000 7f4dd4e74740 10 client.25309 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.322+0000 7f4dd4e74740 10 client.25309 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.322+0000 7f4dd4e74740 20 client.25309 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04ee4cd0 0x562a052eb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4dd4e74740 4 client.25309 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a05245db0 con 0x562a04ee4cd0 2024-09-21T03:02:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04ee4cd0 0x562a052eb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04ee4cd0 0x562a052eb120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3f1c7605ced1b3a server_cookie=615ed4752c22f4f1 in_seq=0 out_seq=0 2024-09-21T03:02:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.323+0000 7f4da77fe640 10 client.25309 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.324+0000 7f4da77fe640 1 -- 172.21.15.16:0/3340639865 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562a05245db0 con 0x562a04ee4cd0 2024-09-21T03:02:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.324+0000 7f4da77fe640 10 client.25309 handle_command_reply: tid=0 2024-09-21T03:02:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.324+0000 7f4dd4e74740 1 client.25309 shutdown 2024-09-21T03:02:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.325+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 msgr2=0x562a052d8fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.325+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0303f0 tx=0x7f4dbc004060 comp rx=0 tx=0).stop 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 shutdown_connections 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04ee4cd0 0x562a052eb120 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052d94f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f26350 0x562a052e4f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/3340639865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04efd490 0x562a052d8fb0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 >> 172.21.15.16:0/3340639865 conn(0x562a04f1a740 msgr2=0x562a052d0630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4da77fe640 0 client.25309 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:27.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 shutdown_connections 2024-09-21T03:02:27.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.327+0000 7f4dd4e74740 1 -- 172.21.15.16:0/3340639865 wait complete. 2024-09-21T03:02:27.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.328+0000 7f4dd4e74740 20 client.25309 trim_cache size 0 max 16384 2024-09-21T03:02:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.348+0000 7f4dd4e74740 1 Processor -- start 2024-09-21T03:02:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.348+0000 7f4dd4e74740 1 -- start start 2024-09-21T03:02:27.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04efd490 0x562a052d8640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052d5fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c4d30 con 0x562a04f05670 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c7560 con 0x562a04efd490 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dd4e74740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c5350 con 0x562a04f26350 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34500/0 (socket says 172.21.15.16:34500) 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 -- 172.21.15.16:0/2714594011 learned_addr learned my addr 172.21.15.16:0/2714594011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04efd490 0x562a052d8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052d5fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 -- 172.21.15.16:0/2714594011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 msgr2=0x562a052d5fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052d5fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 -- 172.21.15.16:0/2714594011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04efd490 msgr2=0x562a052d8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04efd490 0x562a052d8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.349+0000 7f4dc77fe640 1 -- 172.21.15.16:0/2714594011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a052da070 con 0x562a04f05670 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.350+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4db80ad130 tx=0x7f4db8000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db0691a5ac95e74 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.350+0000 7f4dadffb640 1 -- 172.21.15.16:0/2714594011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db801d040 con 0x562a04f05670 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.350+0000 7f4dadffb640 1 -- 172.21.15.16:0/2714594011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db80accb0 con 0x562a04f05670 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.350+0000 7f4dadffb640 1 -- 172.21.15.16:0/2714594011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db80e4c70 con 0x562a04f05670 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 -- 172.21.15.16:0/2714594011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 msgr2=0x562a052ca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f4db80ad130 tx=0x7f4db8000bc0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 -- 172.21.15.16:0/2714594011 shutdown_connections 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04f05670 0x562a052ca0b0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052d5fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/2714594011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04efd490 0x562a052d8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 -- 172.21.15.16:0/2714594011 >> 172.21.15.16:0/2714594011 conn(0x562a04f1a740 msgr2=0x562a052c99a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 -- 172.21.15.16:0/2714594011 shutdown_connections 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.351+0000 7f4dd4e74740 1 -- 172.21.15.16:0/2714594011 wait complete. 2024-09-21T03:02:27.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.352+0000 7f4dd4e74740 1 -- start start 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.352+0000 7f4dd4e74740 1 Processor -- start 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.352+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.352+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.352+0000 7f4dd4e74740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052dd8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34502/0 (socket says 172.21.15.16:34502) 2024-09-21T03:02:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53296/0 (socket says 172.21.15.16:53296) 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc7fff640 1 -- 172.21.15.16:0/4292184227 learned_addr learned my addr 172.21.15.16:0/4292184227 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c5a20 con 0x562a04efd490 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c6e90 con 0x562a04f05670 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4dc81c60f0 con 0x562a04f26350 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dcce9d640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052dd8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dd4e74740 10 client.0 _ll_register_callbacks cb 0x7f4dbc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 -- 172.21.15.16:0/4292184227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 msgr2=0x562a052dd8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052dd8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 -- 172.21.15.16:0/4292184227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 msgr2=0x562a052dce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a05354ae0 con 0x562a04f05670 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4dc77fe640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4db80aa410 tx=0x7f4db8003840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ef9d0a491fac1c05 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.353+0000 7f4da77fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4db801d040 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4db80a9410 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4db80accb0 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4db80acf70 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 10 client.25379 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562a05355d40 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4db80accb0 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4db8058ae0 con 0x562a04f05670 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 10 client.25379 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 10 client.25379 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 20 client.25379 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.354+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04eba080 0x562a05358310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.355+0000 7f4dd4e74740 4 client.25379 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:02:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.355+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562a05241860 con 0x562a04eba080 2024-09-21T03:02:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.355+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04eba080 0x562a05358310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.355+0000 7f4da77fe640 10 client.25379 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:27.359 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.355+0000 7f4dc7fff640 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04eba080 0x562a05358310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42c68b63e6b9805a server_cookie=3b21bbb45ace5fa1 in_seq=0 out_seq=0 2024-09-21T03:02:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.394+0000 7f4da77fe640 1 -- 172.21.15.16:0/4292184227 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562a05241860 con 0x562a04eba080 2024-09-21T03:02:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.394+0000 7f4da77fe640 10 client.25379 handle_command_reply: tid=0 2024-09-21T03:02:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.394+0000 7f4dd4e74740 1 client.25379 shutdown 2024-09-21T03:02:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 msgr2=0x562a052dd3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f4db80aa410 tx=0x7f4db8003840 comp rx=0 tx=0).stop 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 shutdown_connections 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562a04eba080 0x562a05358310 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562a04f26350 0x562a052dd8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562a04f05670 0x562a052dd3a0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 --2- 172.21.15.16:0/4292184227 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562a04efd490 0x562a052dce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 >> 172.21.15.16:0/4292184227 conn(0x562a04f1a740 msgr2=0x562a052ce460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4da77fe640 0 client.25379 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 shutdown_connections 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 1 -- 172.21.15.16:0/4292184227 wait complete. 2024-09-21T03:02:27.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.395+0000 7f4dd4e74740 20 client.25379 trim_cache size 0 max 16384 2024-09-21T03:02:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:02:27.398 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:02:27.398 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "e2f8ab2e-228d-4686-8d9f-164a925af15d", 2024-09-21T03:02:27.398 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:02:27.398 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 msgr2=0x7f4db00b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 0x7f4db00b8c90 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f4dc001b000 tx=0x7f4dc000c870 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 msgr2=0x7f4dc8115680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 0x7f4dc8115680 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806c6e0 tx=0x7f4db800cdd0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 shutdown_connections 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4db00b6780 0x7f4db00b8c90 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4dc8111f40 0x7f4dc8115680 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc811cc30 0x7f4dc8115bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 --2- 172.21.15.16:0/460982609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4dc810a9e0 0x7f4dc811c3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 >> 172.21.15.16:0/460982609 conn(0x7f4dc8070ec0 msgr2=0x7f4dc8073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 shutdown_connections 2024-09-21T03:02:27.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:27.396+0000 7f4dce900640 1 -- 172.21.15.16:0/460982609 wait complete. 2024-09-21T03:02:28.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:27 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:02:28.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:27 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.160 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:27 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:02:28.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:27 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:02:28.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:27 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:27 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:02:28.189 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:02:28.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:27 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:02:28.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:27 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:28.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:27 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:02:28.448 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:02:29.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:28 smithi016 ceph-mon[31253]: pgmap v384: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 407 MiB/s wr, 1.54k op/s 2024-09-21T03:02:29.166 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:28 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:28 smithi018 ceph-mon[34587]: pgmap v384: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 407 MiB/s wr, 1.54k op/s 2024-09-21T03:02:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:28 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:28 smithi123 ceph-mon[34645]: pgmap v384: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1.0 MiB/s rd, 407 MiB/s wr, 1.54k op/s 2024-09-21T03:02:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:28 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.302+0000 7f50c63ca640 1 -- 172.21.15.16:0/2084636826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c010a1b0 msgr2=0x7f50c010a5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.302+0000 7f50c63ca640 1 --2- 172.21.15.16:0/2084636826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c010a1b0 0x7f50c010a5c0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f50b4009900 tx=0x7f50b402a440 comp rx=0 tx=0).stop 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 -- 172.21.15.16:0/2084636826 shutdown_connections 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 --2- 172.21.15.16:0/2084636826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50c0110440 0x7f50c0116c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 --2- 172.21.15.16:0/2084636826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c010faf0 0x7f50c010ff00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 --2- 172.21.15.16:0/2084636826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c010a1b0 0x7f50c010a5c0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 -- 172.21.15.16:0/2084636826 >> 172.21.15.16:0/2084636826 conn(0x7f50c00780e0 msgr2=0x7f50c01081e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.303+0000 7f50c63ca640 1 -- 172.21.15.16:0/2084636826 shutdown_connections 2024-09-21T03:02:29.306 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.304+0000 7f50c63ca640 1 -- 172.21.15.16:0/2084636826 wait complete. 2024-09-21T03:02:29.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 Processor -- start 2024-09-21T03:02:29.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 -- start start 2024-09-21T03:02:29.307 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50c010a1b0 0x7f50c01a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c010faf0 0x7f50c01a1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50c011a230 con 0x7f50c0110440 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.305+0000 7f50c63ca640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50c011a0b0 con 0x7f50c010faf0 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.306+0000 7f50c63ca640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50c011a3b0 con 0x7f50c010a1b0 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.306+0000 7f50c4967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.306+0000 7f50c4967640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34528/0 (socket says 172.21.15.16:34528) 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.306+0000 7f50c4967640 1 -- 172.21.15.16:0/553412716 learned_addr learned my addr 172.21.15.16:0/553412716 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:29.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.307+0000 7f50c4967640 1 -- 172.21.15.16:0/553412716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50c010a1b0 msgr2=0x7f50c01a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.307+0000 7f50c4967640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50c010a1b0 0x7f50c01a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.307+0000 7f50c4967640 1 -- 172.21.15.16:0/553412716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c010faf0 msgr2=0x7f50c01a1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.307+0000 7f50c4967640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c010faf0 0x7f50c01a1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.307+0000 7f50c4967640 1 -- 172.21.15.16:0/553412716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f50c01a6bf0 con 0x7f50c0110440 2024-09-21T03:02:29.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.308+0000 7f50c4967640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50b800b990 tx=0x7f50b800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.309+0000 7f50bd7fa640 1 -- 172.21.15.16:0/553412716 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8019070 con 0x7f50c0110440 2024-09-21T03:02:29.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.309+0000 7f50bd7fa640 1 -- 172.21.15.16:0/553412716 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b800cba0 con 0x7f50c0110440 2024-09-21T03:02:29.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.309+0000 7f50bd7fa640 1 -- 172.21.15.16:0/553412716 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8019070 con 0x7f50c0110440 2024-09-21T03:02:29.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.309+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f50c01a6ee0 con 0x7f50c0110440 2024-09-21T03:02:29.311 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.310+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50c01ad640 con 0x7f50c0110440 2024-09-21T03:02:29.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.317+0000 7f50bd7fa640 1 -- 172.21.15.16:0/553412716 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f50b801c070 con 0x7f50c0110440 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.321+0000 7f50bd7fa640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 0x7f50a80b8f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.322+0000 7f50bd7fa640 1 -- 172.21.15.16:0/553412716 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f50b800c9a0 con 0x7f50c0110440 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.323+0000 7f50cc93e740 1 Processor -- start 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 -- start start 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50bffff640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 0x7f50a80b8f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a50b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a510d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf59c6e50 con 0x55faf56a9070 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55faf59b6280 con 0x55faf5688390 2024-09-21T03:02:29.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.324+0000 7f50cc93e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55faf59ba090 con 0x55faf56801b0 2024-09-21T03:02:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.325+0000 7f50bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.325+0000 7f50bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34544/0 (socket says 172.21.15.16:34544) 2024-09-21T03:02:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.325+0000 7f50bffff640 1 -- 172.21.15.16:0/910041909 learned_addr learned my addr 172.21.15.16:0/910041909 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:29.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.326+0000 7f50bffff640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 0x7f50a80b8f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005f70 tx=0x7f50b4031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.326+0000 7f50bf7fe640 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a50b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.326+0000 7f50bffff640 1 -- 172.21.15.16:0/910041909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 msgr2=0x55faf5a50b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.326+0000 7f50bffff640 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a50b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.327+0000 7f50bffff640 1 -- 172.21.15.16:0/910041909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 msgr2=0x55faf5a510d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:02:29.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.327+0000 7f50bffff640 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a510d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.327+0000 7f50bffff640 1 -- 172.21.15.16:0/910041909 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf5714650 con 0x55faf56a9070 2024-09-21T03:02:29.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.327+0000 7f50bffff640 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50b4004400 tx=0x7f50b4042000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70144753465e13d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.329+0000 7f50a67fc640 1 -- 172.21.15.16:0/910041909 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4035040 con 0x55faf56a9070 2024-09-21T03:02:29.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.329+0000 7f50a67fc640 1 -- 172.21.15.16:0/910041909 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f50b4003d70 con 0x55faf56a9070 2024-09-21T03:02:29.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.330+0000 7f50cc93e740 1 -- 172.21.15.16:0/910041909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 msgr2=0x55faf5a50650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.330+0000 7f50cc93e740 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f50b4004400 tx=0x7f50b4042000 comp rx=0 tx=0).stop 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.334+0000 7f50cc93e740 1 -- 172.21.15.16:0/910041909 shutdown_connections 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.334+0000 7f50cc93e740 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a510d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.334+0000 7f50cc93e740 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a50b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.334+0000 7f50cc93e740 1 --2- 172.21.15.16:0/910041909 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a50650 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.334+0000 7f50cc93e740 1 -- 172.21.15.16:0/910041909 >> 172.21.15.16:0/910041909 conn(0x55faf569d460 msgr2=0x55faf5a4e540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.335+0000 7f50cc93e740 1 -- 172.21.15.16:0/910041909 shutdown_connections 2024-09-21T03:02:29.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.335+0000 7f50cc93e740 1 -- 172.21.15.16:0/910041909 wait complete. 2024-09-21T03:02:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.336+0000 7f50cc93e740 1 -- start start 2024-09-21T03:02:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.336+0000 7f50cc93e740 1 Processor -- start 2024-09-21T03:02:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.337+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a5c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a61a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50cc93e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55faf5817700 con 0x55faf56a9070 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50cc93e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55faf5951110 con 0x55faf5688390 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50cc93e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55faf5a56c70 con 0x55faf56801b0 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50bf7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53334/0 (socket says 172.21.15.16:53334) 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.338+0000 7f50bf7fe640 1 -- 172.21.15.16:0/81730918 learned_addr learned my addr 172.21.15.16:0/81730918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:29.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.339+0000 7f50bffff640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a5c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.339+0000 7f50cc93e740 10 client.0 _ll_register_callbacks cb 0x55faf596a990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 -- 172.21.15.16:0/81730918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 msgr2=0x55faf5a5c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a5c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 -- 172.21.15.16:0/81730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 msgr2=0x55faf5a61a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a61a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 -- 172.21.15.16:0/81730918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf597ec40 con 0x55faf5688390 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f50bf7fe640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50b000bfd0 tx=0x7f50b000d5f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c09a51b2246d0981 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.340+0000 7f509f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f509f7fe640 1 -- 172.21.15.16:0/81730918 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b000de20 con 0x55faf5688390 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f50b4003d70 con 0x55faf5688390 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f509f7fe640 1 -- 172.21.15.16:0/81730918 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f50b00125f0 con 0x55faf5688390 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f50bffff640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a5c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f509f7fe640 1 -- 172.21.15.16:0/81730918 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f50b00127f0 con 0x55faf5688390 2024-09-21T03:02:29.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.341+0000 7f50cc93e740 10 client.25382 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.342+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55faf5a6a980 con 0x55faf5688390 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.342+0000 7f509f7fe640 1 -- 172.21.15.16:0/81730918 <== mon.2 v2:172.21.15.18:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f50b0013070 con 0x55faf5688390 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 10 client.25382 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 10 client.25382 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 20 client.25382 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf56679f0 0x55faf5a6dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 4 client.25382 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:02:29.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.343+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55faf59c8da0 con 0x55faf56679f0 2024-09-21T03:02:29.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.345+0000 7f50bffff640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf56679f0 0x55faf5a6dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.352+0000 7f509f7fe640 10 client.25382 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:29.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.352+0000 7f50bffff640 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf56679f0 0x55faf5a6dd80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8125674bf8adce56 server_cookie=7522b5b29f049d7e in_seq=0 out_seq=0 2024-09-21T03:02:29.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.353+0000 7f509f7fe640 1 -- 172.21.15.16:0/81730918 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55faf59c8da0 con 0x55faf56679f0 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.353+0000 7f509f7fe640 10 client.25382 handle_command_reply: tid=0 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.354+0000 7f50cc93e740 1 client.25382 shutdown 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.354+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 msgr2=0x55faf5a5ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.354+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f50b000bfd0 tx=0x7f50b000d5f0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.354+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 shutdown_connections 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf56679f0 0x55faf5a6dd80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56a9070 0x55faf5a61a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f509f7fe640 0 client.25382 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf5688390 0x55faf5a5ca40 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 --2- 172.21.15.16:0/81730918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56801b0 0x55faf5a5c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 >> 172.21.15.16:0/81730918 conn(0x55faf569d460 msgr2=0x55faf5a4cb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.355+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 shutdown_connections 2024-09-21T03:02:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.356+0000 7f50cc93e740 1 -- 172.21.15.16:0/81730918 wait complete. 2024-09-21T03:02:29.360 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.356+0000 7f50cc93e740 20 client.25382 trim_cache size 0 max 16384 2024-09-21T03:02:29.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.400+0000 7f50cc93e740 1 Processor -- start 2024-09-21T03:02:29.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.400+0000 7f50cc93e740 1 -- start start 2024-09-21T03:02:29.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.401+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34570/0 (socket says 172.21.15.16:34570) 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.401+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf5688390 0x55faf5a4e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50bffff640 1 -- 172.21.15.16:0/2303461022 learned_addr learned my addr 172.21.15.16:0/2303461022 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50cc93e740 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56a9070 0x55faf5a59470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00c7e0 con 0x55faf56801b0 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00f060 con 0x55faf56a9070 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.402+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00ce50 con 0x55faf5688390 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.403+0000 7f50bffff640 1 -- 172.21.15.16:0/2303461022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf5688390 msgr2=0x55faf5a4e460 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.403+0000 7f50bffff640 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf5688390 0x55faf5a4e460 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.403+0000 7f50bffff640 1 -- 172.21.15.16:0/2303461022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56a9070 msgr2=0x55faf5a59470 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.403+0000 7f50bffff640 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56a9070 0x55faf5a59470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.403+0000 7f50bffff640 1 -- 172.21.15.16:0/2303461022 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55faf5a5b1e0 con 0x55faf56801b0 2024-09-21T03:02:29.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.404+0000 7f50bffff640 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50b40043d0 tx=0x7f50b4043000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b5314cc89aa5ef2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.407+0000 7f50a5ffb640 1 -- 172.21.15.16:0/2303461022 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4035040 con 0x55faf56801b0 2024-09-21T03:02:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.407+0000 7f50a5ffb640 1 -- 172.21.15.16:0/2303461022 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55faf59c8da0 con 0x55faf56801b0 2024-09-21T03:02:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.407+0000 7f50a5ffb640 1 -- 172.21.15.16:0/2303461022 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b4031de0 con 0x55faf56801b0 2024-09-21T03:02:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.408+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 msgr2=0x55faf5a49280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.408+0000 7f50cc93e740 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f50b40043d0 tx=0x7f50b4043000 comp rx=0 tx=0).stop 2024-09-21T03:02:29.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.412+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 shutdown_connections 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.412+0000 7f50cc93e740 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56a9070 0x55faf5a59470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.412+0000 7f50cc93e740 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf5688390 0x55faf5a4e460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.412+0000 7f50cc93e740 1 --2- 172.21.15.16:0/2303461022 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf56801b0 0x55faf5a49280 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.412+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 >> 172.21.15.16:0/2303461022 conn(0x55faf569d460 msgr2=0x55faf5a5c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.413+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 shutdown_connections 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.413+0000 7f50cc93e740 1 -- 172.21.15.16:0/2303461022 wait complete. 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.414+0000 7f50cc93e740 1 -- start start 2024-09-21T03:02:29.416 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.414+0000 7f50cc93e740 1 Processor -- start 2024-09-21T03:02:29.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.415+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 0x55faf5a68890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.415+0000 7f50cc93e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf5688390 0x55faf5a62d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.415+0000 7f50bffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 0x55faf5a68890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50bffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 0x55faf5a68890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53344/0 (socket says 172.21.15.16:53344) 2024-09-21T03:02:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50bffff640 1 -- 172.21.15.16:0/243816734 learned_addr learned my addr 172.21.15.16:0/243816734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:02:29.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 0x55faf5a63420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00d520 con 0x55faf5688390 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00e990 con 0x55faf56801b0 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f50ac00dbf0 con 0x55faf56a9070 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50c4967640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 0x55faf5a63420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.416+0000 7f50cc93e740 10 client.0 _ll_register_callbacks cb 0x7f50b000b100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.418+0000 7f50c4967640 1 -- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 msgr2=0x55faf5a68890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.418+0000 7f50c4967640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 0x55faf5a68890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.418+0000 7f50c4967640 1 -- 172.21.15.16:0/243816734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf5688390 msgr2=0x55faf5a62d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.418+0000 7f50c4967640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf5688390 0x55faf5a62d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.421 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.419+0000 7f50c4967640 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55faf5af9cc0 con 0x55faf56a9070 2024-09-21T03:02:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.420+0000 7f50c4967640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 0x55faf5a63420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50b80e3ba0 tx=0x7f50b80abd20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d78c154efd71383 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:02:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.420+0000 7f509f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:02:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.420+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f50b8019070 con 0x55faf56a9070 2024-09-21T03:02:29.423 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.420+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f50b80e3e20 con 0x55faf56a9070 2024-09-21T03:02:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.422+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55faf59c8da0 con 0x55faf56a9070 2024-09-21T03:02:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.424+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f50b80b0d80 con 0x55faf56a9070 2024-09-21T03:02:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.431+0000 7f50cc93e740 10 client.25324 fetch_fsmap learned FSMap version 20 2024-09-21T03:02:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.431+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55faf5afae00 con 0x55faf56a9070 2024-09-21T03:02:29.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55faf59c8da0 con 0x55faf56a9070 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f50b80aaf70 con 0x55faf56a9070 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 10 client.25324 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 10 client.25324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 20 client.25324 populate_metadata read hostname 'smithi016' 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf563cda0 0x55faf5afd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 4 client.25324 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.433+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55faf59c4850 con 0x55faf563cda0 2024-09-21T03:02:29.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.435+0000 7f50bffff640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf563cda0 0x55faf5afd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:02:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.440+0000 7f50bffff640 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf563cda0 0x55faf5afd570 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d830c95af527338d server_cookie=f966c07aac366684 in_seq=0 out_seq=0 2024-09-21T03:02:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.440+0000 7f509f7fe640 10 client.25324 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.440+0000 7f509f7fe640 1 -- 172.21.15.16:0/243816734 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1706 (crc 0 0 0) 0x55faf59c4850 con 0x55faf563cda0 2024-09-21T03:02:29.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.440+0000 7f509f7fe640 10 client.25324 handle_command_reply: tid=0 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.440+0000 7f50cc93e740 1 client.25324 shutdown 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 msgr2=0x55faf5a63420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 0x55faf5a63420 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f50b80e3ba0 tx=0x7f50b80abd20 comp rx=0 tx=0).stop 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 shutdown_connections 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55faf563cda0 0x55faf5afd570 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55faf5688390 0x55faf5a62d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f509f7fe640 0 client.25324 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55faf56a9070 0x55faf5a63420 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 --2- 172.21.15.16:0/243816734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55faf56801b0 0x55faf5a68890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.442+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 >> 172.21.15.16:0/243816734 conn(0x55faf569d460 msgr2=0x55faf5a557d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.444+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 shutdown_connections 2024-09-21T03:02:29.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.444+0000 7f50cc93e740 1 -- 172.21.15.16:0/243816734 wait complete. 2024-09-21T03:02:29.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.444+0000 7f50cc93e740 20 client.25324 trim_cache size 0 max 16384 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "e2f8ab2e-228d-4686-8d9f-164a925af15d", 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:29.451 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.452 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:29.453 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:29.454 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:02:29.455 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:02:29.456 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:02:29.456 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:02:29.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:02:29.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:02:29.456 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.450+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 msgr2=0x7f50a80b8f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.451+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 0x7f50a80b8f90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f50b4005f70 tx=0x7f50b4031400 comp rx=0 tx=0).stop 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.451+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 msgr2=0x7f50c01a64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.451+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f50b800b990 tx=0x7f50b800be50 comp rx=0 tx=0).stop 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 shutdown_connections 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f50a80b6a80 0x7f50a80b8f90 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f50c0110440 0x7f50c01a64c0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f50c010faf0 0x7f50c01a1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 --2- 172.21.15.16:0/553412716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50c010a1b0 0x7f50c01a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:02:29.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.453+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 >> 172.21.15.16:0/553412716 conn(0x7f50c00780e0 msgr2=0x7f50c0078970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:02:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.459+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 shutdown_connections 2024-09-21T03:02:29.466 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:02:29.459+0000 7f50c63ca640 1 -- 172.21.15.16:0/553412716 wait complete. 2024-09-21T03:02:30.484 INFO:tasks.cephfs.filesystem:scrub status for tag:e2f8ab2e-228d-4686-8d9f-164a925af15d - {'path': '/', 'tag': 'e2f8ab2e-228d-4686-8d9f-164a925af15d', 'options': 'recursive,force'} 2024-09-21T03:02:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:30 smithi018 ceph-mon[34587]: pgmap v385: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 418 MiB/s wr, 1.45k op/s 2024-09-21T03:02:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:30 smithi123 ceph-mon[34645]: pgmap v385: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 418 MiB/s wr, 1.45k op/s 2024-09-21T03:02:31.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:30 smithi016 ceph-mon[31253]: pgmap v385: 97 pgs: 97 active+clean; 14 GiB data, 42 GiB used, 1.0 TiB / 1.0 TiB avail; 1002 KiB/s rd, 418 MiB/s wr, 1.45k op/s 2024-09-21T03:02:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:32 smithi018 ceph-mon[34587]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 463 MiB/s wr, 1.46k op/s 2024-09-21T03:02:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:32 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:02:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:32 smithi123 ceph-mon[34645]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 463 MiB/s wr, 1.46k op/s 2024-09-21T03:02:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:32 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:02:33.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:32 smithi016 ceph-mon[31253]: pgmap v386: 97 pgs: 97 active+clean; 14 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 463 MiB/s wr, 1.46k op/s 2024-09-21T03:02:33.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:32 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:02:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:34 smithi123 ceph-mon[34645]: pgmap v387: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 423 MiB/s wr, 1.04k op/s 2024-09-21T03:02:35.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:34 smithi016 ceph-mon[31253]: pgmap v387: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 423 MiB/s wr, 1.04k op/s 2024-09-21T03:02:35.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:34 smithi018 ceph-mon[34587]: pgmap v387: 97 pgs: 97 active+clean; 15 GiB data, 44 GiB used, 1.0 TiB / 1.0 TiB avail; 1.3 MiB/s rd, 423 MiB/s wr, 1.04k op/s 2024-09-21T03:02:36.333 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:36 smithi018 ceph-mon[34587]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 573 MiB/s wr, 1.57k op/s 2024-09-21T03:02:36.334 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:36 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:36.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:36 smithi123 ceph-mon[34645]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 573 MiB/s wr, 1.57k op/s 2024-09-21T03:02:36.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:36 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:36.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:36 smithi016 ceph-mon[31253]: pgmap v388: 97 pgs: 97 active+clean; 16 GiB data, 49 GiB used, 1024 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 573 MiB/s wr, 1.57k op/s 2024-09-21T03:02:36.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:36 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:02:37.668 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:37.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:37.669 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:02:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:37.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:02:37.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:02:37.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:02:37.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:02:38.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:38 smithi018 ceph-mon[34587]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.52k op/s 2024-09-21T03:02:38.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:38 smithi123 ceph-mon[34645]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.52k op/s 2024-09-21T03:02:38.768 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:38 smithi016 ceph-mon[31253]: pgmap v389: 97 pgs: 97 active+clean; 17 GiB data, 51 GiB used, 1022 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.52k op/s 2024-09-21T03:02:40.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:40 smithi018 ceph-mon[34587]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.44k op/s 2024-09-21T03:02:40.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:40 smithi123 ceph-mon[34645]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.44k op/s 2024-09-21T03:02:40.769 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:40 smithi016 ceph-mon[31253]: pgmap v390: 97 pgs: 97 active+clean; 18 GiB data, 53 GiB used, 1020 GiB / 1.0 TiB avail; 1009 KiB/s rd, 450 MiB/s wr, 1.44k op/s 2024-09-21T03:02:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:42 smithi018 ceph-mon[34587]: pgmap v391: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 MiB/s wr, 1.37k op/s 2024-09-21T03:02:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:42 smithi123 ceph-mon[34645]: pgmap v391: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 MiB/s wr, 1.37k op/s 2024-09-21T03:02:43.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:42 smithi016 ceph-mon[31253]: pgmap v391: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 613 MiB/s wr, 1.37k op/s 2024-09-21T03:02:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:44 smithi123 ceph-mon[34645]: pgmap v392: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 540 MiB/s wr, 942 op/s 2024-09-21T03:02:45.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:44 smithi016 ceph-mon[31253]: pgmap v392: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 540 MiB/s wr, 942 op/s 2024-09-21T03:02:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:44 smithi018 ceph-mon[34587]: pgmap v392: 97 pgs: 97 active+clean; 20 GiB data, 60 GiB used, 1013 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 540 MiB/s wr, 942 op/s 2024-09-21T03:02:47.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:46 smithi016 ceph-mon[31253]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 655 MiB/s wr, 1.55k op/s 2024-09-21T03:02:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:46 smithi018 ceph-mon[34587]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 655 MiB/s wr, 1.55k op/s 2024-09-21T03:02:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:46 smithi123 ceph-mon[34645]: pgmap v393: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 655 MiB/s wr, 1.55k op/s 2024-09-21T03:02:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:48 smithi018 ceph-mon[34587]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 527 MiB/s wr, 1.25k op/s 2024-09-21T03:02:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:48 smithi123 ceph-mon[34645]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 527 MiB/s wr, 1.25k op/s 2024-09-21T03:02:49.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:48 smithi016 ceph-mon[31253]: pgmap v394: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 527 MiB/s wr, 1.25k op/s 2024-09-21T03:02:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:50 smithi018 ceph-mon[34587]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 495 MiB/s wr, 1.18k op/s 2024-09-21T03:02:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:50 smithi123 ceph-mon[34645]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 495 MiB/s wr, 1.18k op/s 2024-09-21T03:02:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:50 smithi016 ceph-mon[31253]: pgmap v395: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 495 MiB/s wr, 1.18k op/s 2024-09-21T03:02:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:52 smithi018 ceph-mon[34587]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 MiB/s wr, 1.24k op/s 2024-09-21T03:02:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:52 smithi123 ceph-mon[34645]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 MiB/s wr, 1.24k op/s 2024-09-21T03:02:53.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:52 smithi016 ceph-mon[31253]: pgmap v396: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 566 MiB/s wr, 1.24k op/s 2024-09-21T03:02:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:54 smithi018 ceph-mon[34587]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 381 MiB/s wr, 1.13k op/s 2024-09-21T03:02:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:54 smithi123 ceph-mon[34645]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 381 MiB/s wr, 1.13k op/s 2024-09-21T03:02:55.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:54 smithi016 ceph-mon[31253]: pgmap v397: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 381 MiB/s wr, 1.13k op/s 2024-09-21T03:02:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:56 smithi018 ceph-mon[34587]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 447 MiB/s wr, 1.12k op/s 2024-09-21T03:02:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:56 smithi123 ceph-mon[34645]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 447 MiB/s wr, 1.12k op/s 2024-09-21T03:02:57.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:56 smithi016 ceph-mon[31253]: pgmap v398: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 447 MiB/s wr, 1.12k op/s 2024-09-21T03:02:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:02:58 smithi018 ceph-mon[34587]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 331 MiB/s wr, 408 op/s 2024-09-21T03:02:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:02:58 smithi123 ceph-mon[34645]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 331 MiB/s wr, 408 op/s 2024-09-21T03:02:59.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:02:58 smithi016 ceph-mon[31253]: pgmap v399: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 331 MiB/s wr, 408 op/s 2024-09-21T03:03:00.485 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:03:00.901 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:00 smithi018 ceph-mon[34587]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 315 MiB/s wr, 189 op/s 2024-09-21T03:03:01.210 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:00 smithi016 ceph-mon[31253]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 315 MiB/s wr, 189 op/s 2024-09-21T03:03:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:00 smithi123 ceph-mon[34645]: pgmap v400: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 315 MiB/s wr, 189 op/s 2024-09-21T03:03:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 -- 172.21.15.16:0/3387720066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 msgr2=0x7fc1a81098e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3387720066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a81098e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fc198009900 tx=0x7fc19802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 -- 172.21.15.16:0/3387720066 shutdown_connections 2024-09-21T03:03:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3387720066 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a81098e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3387720066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a8110650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3387720066 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a8103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 -- 172.21.15.16:0/3387720066 >> 172.21.15.16:0/3387720066 conn(0x7fc1a80ff5e0 msgr2=0x7fc1a8101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 -- 172.21.15.16:0/3387720066 shutdown_connections 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.481+0000 7fc1afae1640 1 -- 172.21.15.16:0/3387720066 wait complete. 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 Processor -- start 2024-09-21T03:03:01.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 -- start start 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a819c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a819cc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a81a1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8113ae0 con 0x7fc1a81094d0 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8113960 con 0x7fc1a8103b90 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1afae1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a8113c60 con 0x7fc1a8109e20 2024-09-21T03:03:01.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a819cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a819cc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43580/0 (socket says 172.21.15.16:43580) 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1ad07c640 1 -- 172.21.15.16:0/3671538330 learned_addr learned my addr 172.21.15.16:0/3671538330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a819c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.482+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a81a1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 -- 172.21.15.16:0/3671538330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 msgr2=0x7fc1a81a1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a81a1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 -- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 msgr2=0x7fc1a819cc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a819cc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 -- 172.21.15.16:0/3671538330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc1a81a2620 con 0x7fc1a8103b90 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a819c6b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400e9f0 tx=0x7fc1a400eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc19effd640 1 -- 172.21.15.16:0/3671538330 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a400ccc0 con 0x7fc1a8103b90 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc19effd640 1 -- 172.21.15.16:0/3671538330 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc1a40108e0 con 0x7fc1a8103b90 2024-09-21T03:03:01.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc19effd640 1 -- 172.21.15.16:0/3671538330 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a4018880 con 0x7fc1a8103b90 2024-09-21T03:03:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc1a81a2910 con 0x7fc1a8103b90 2024-09-21T03:03:01.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.483+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc1a81a9070 con 0x7fc1a8103b90 2024-09-21T03:03:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 Processor -- start 2024-09-21T03:03:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 -- start start 2024-09-21T03:03:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16ccb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559dd1642e50 con 0x559dd12fc1b0 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559dd1632280 con 0x559dd1304390 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1b6055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559dd1636090 con 0x559dd1325070 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.487+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ae07e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:54160/0 (socket says 172.21.15.16:54160) 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1549969530 learned_addr learned my addr 172.21.15.16:0/1549969530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ae07e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51910/0 (socket says 172.21.15.16:51910) 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc19effd640 1 -- 172.21.15.16:0/3671538330 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc1a4010430 con 0x7fc1a8103b90 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1549969530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 msgr2=0x559dd16cc650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16ccb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1549969530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 msgr2=0x559dd16ccb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16ccb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1549969530 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd1390650 con 0x559dd1304390 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.488+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc198009230 tx=0x7fc198031b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=67216d3cea917caa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc193fff640 1 -- 172.21.15.16:0/1549969530 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198040070 con 0x559dd1304390 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc193fff640 1 -- 172.21.15.16:0/1549969530 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc198004030 con 0x559dd1304390 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc193fff640 1 -- 172.21.15.16:0/1549969530 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198035040 con 0x559dd1304390 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc19effd640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 0x7fc1940b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 0x7fc1940b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.489+0000 7fc19effd640 1 -- 172.21.15.16:0/3671538330 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc1a40e7a70 con 0x7fc1a8103b90 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 -- 172.21.15.16:0/1549969530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 msgr2=0x559dd16cd0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 0x7fc1940b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fc198009230 tx=0x7fc198031b10 comp rx=0 tx=0).stop 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 -- 172.21.15.16:0/1549969530 shutdown_connections 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16ccb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16cd0d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1549969530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16cc650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 -- 172.21.15.16:0/1549969530 >> 172.21.15.16:0/1549969530 conn(0x559dd1319460 msgr2=0x559dd16ca540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 -- 172.21.15.16:0/1549969530 shutdown_connections 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.490+0000 7fc1b6055740 1 -- 172.21.15.16:0/1549969530 wait complete. 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.491+0000 7fc1b6055740 1 -- start start 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.491+0000 7fc1b6055740 1 Processor -- start 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 0x559dd16d8270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 0x559dd16e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x559dd1493700 con 0x559dd1304390 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x559dd15cd110 con 0x559dd1325070 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x559dd16dadd0 con 0x559dd12fc1b0 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ad87d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 0x559dd16d8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ae07e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 0x559dd16e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ad87d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51914/0 (socket says 172.21.15.16:51914) 2024-09-21T03:03:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 0x559dd16d8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43604/0 (socket says 172.21.15.16:43604) 2024-09-21T03:03:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ad87d640 1 -- 172.21.15.16:0/2904533693 learned_addr learned my addr 172.21.15.16:0/2904533693 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:01.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1b6055740 10 client.0 _ll_register_callbacks cb 0x559dd15e6990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ae07e640 1 -- 172.21.15.16:0/2904533693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 msgr2=0x559dd16d7d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ae07e640 1 -- 172.21.15.16:0/2904533693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 msgr2=0x559dd16d8270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.492+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 0x559dd16d8270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1ae07e640 1 -- 172.21.15.16:0/2904533693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd15fac40 con 0x559dd1325070 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:01.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 0x559dd16e40c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0011590 tx=0x7fc1a0004ab0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d7c48e09f3aa120 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1917fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1a002c060 con 0x559dd1325070 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc1a0017030 con 0x559dd1325070 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc198004030 con 0x559dd1325070 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc1a0011d60 con 0x559dd1325070 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1b6055740 10 client.25397 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.493+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559dd16e6ac0 con 0x559dd1325070 2024-09-21T03:03:01.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc198004030 con 0x559dd1325070 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc1a001c3f0 con 0x559dd1325070 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 10 client.25397 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 10 client.25397 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 20 client.25397 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd12e39f0 0x559dd16ea070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 4 client.25397 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dd1644da0 con 0x559dd12e39f0 2024-09-21T03:03:01.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.494+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd12e39f0 0x559dd16ea070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.495+0000 7fc1917fa640 10 client.25397 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.495+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd12e39f0 0x559dd16ea070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a7c1b17a9c1943 server_cookie=b9bdf4f8e54706b1 in_seq=0 out_seq=0 2024-09-21T03:03:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1917fa640 1 -- 172.21.15.16:0/2904533693 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x559dd1644da0 con 0x559dd12e39f0 2024-09-21T03:03:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1917fa640 10 client.25397 handle_command_reply: tid=0 2024-09-21T03:03:01.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 client.25397 shutdown 2024-09-21T03:03:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 msgr2=0x559dd16e40c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 0x559dd16e40c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0011590 tx=0x7fc1a0004ab0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 shutdown_connections 2024-09-21T03:03:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd12e39f0 0x559dd16ea070 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1304390 0x559dd16d8270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1325070 0x559dd16e40c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2904533693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12fc1b0 0x559dd16d7d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 >> 172.21.15.16:0/2904533693 conn(0x559dd1319460 msgr2=0x559dd16cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1917fa640 0 client.25397 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 shutdown_connections 2024-09-21T03:03:01.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 1 -- 172.21.15.16:0/2904533693 wait complete. 2024-09-21T03:03:01.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.496+0000 7fc1b6055740 20 client.25397 trim_cache size 0 max 16384 2024-09-21T03:03:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.526+0000 7fc1b6055740 1 Processor -- start 2024-09-21T03:03:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.526+0000 7fc1b6055740 1 -- start start 2024-09-21T03:03:01.530 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.526+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16d9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.526+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16c9b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.526+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16ca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ae07e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16ca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ae07e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16ca0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51920/0 (socket says 172.21.15.16:51920) 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ae07e640 1 -- 172.21.15.16:0/1939550188 learned_addr learned my addr 172.21.15.16:0/1939550188 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:01.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16c9b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81af8e0 con 0x559dd12fc1b0 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b2160 con 0x559dd1304390 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81aff50 con 0x559dd1325070 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1939550188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 msgr2=0x559dd16ca0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16ca0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1939550188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 msgr2=0x559dd16d9960 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:03:01.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16d9960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.527+0000 7fc1ad07c640 1 -- 172.21.15.16:0/1939550188 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dd16ce940 con 0x559dd1304390 2024-09-21T03:03:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.530+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16c9b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1980052a0 tx=0x7fc1980051a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6f23250c1c1f93b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.530+0000 7fc1937fe640 1 -- 172.21.15.16:0/1939550188 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198040070 con 0x559dd1304390 2024-09-21T03:03:01.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.530+0000 7fc1937fe640 1 -- 172.21.15.16:0/1939550188 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc198005440 con 0x559dd1304390 2024-09-21T03:03:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.532+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 msgr2=0x559dd16c9b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.532+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16c9b70 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1980052a0 tx=0x7fc1980051a0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.534+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 shutdown_connections 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.534+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd1325070 0x559dd16ca0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.534+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd1304390 0x559dd16c9b70 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fc1980052a0 tx=0x7fc1980051a0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.534+0000 7fc1b6055740 1 --2- 172.21.15.16:0/1939550188 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd12fc1b0 0x559dd16d9960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.534+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 >> 172.21.15.16:0/1939550188 conn(0x559dd1319460 msgr2=0x559dd16d9250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.539+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 shutdown_connections 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.541+0000 7fc1b6055740 1 -- 172.21.15.16:0/1939550188 wait complete. 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.541+0000 7fc1b6055740 1 -- start start 2024-09-21T03:03:01.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.542+0000 7fc1b6055740 1 Processor -- start 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.543+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd12fc1b0 0x559dd16dc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.543+0000 7fc1b6055740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1ad07c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43612/0 (socket says 172.21.15.16:43612) 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1ad07c640 1 -- 172.21.15.16:0/2589793610 learned_addr learned my addr 172.21.15.16:0/2589793610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12b8da0 0x559dd16dcb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b0620 con 0x559dd1325070 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b1a90 con 0x559dd12fc1b0 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc1a81b0cf0 con 0x559dd12b8da0 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1b6055740 10 client.0 _ll_register_callbacks cb 0x7fc1a0011440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:01.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.544+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd12fc1b0 0x559dd16dc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ae07e640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12b8da0 0x559dd16dcb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 -- 172.21.15.16:0/2589793610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12b8da0 msgr2=0x559dd16dcb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12b8da0 0x559dd16dcb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 -- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd12fc1b0 msgr2=0x559dd16dc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd12fc1b0 0x559dd16dc0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dd175c350 con 0x559dd1325070 2024-09-21T03:03:01.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.545+0000 7fc1ad07c640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc198005940 tx=0x7fc198009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91974151fc283896 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.546+0000 7fc1917fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:03:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.546+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc198040040 con 0x559dd1325070 2024-09-21T03:03:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.546+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fc19804a040 con 0x559dd1325070 2024-09-21T03:03:01.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.547+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc198005440 con 0x559dd1325070 2024-09-21T03:03:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.547+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc198035040 con 0x559dd1325070 2024-09-21T03:03:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.547+0000 7fc1b6055740 10 client.15702 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:01.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.547+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x559dd175d5c0 con 0x559dd1325070 2024-09-21T03:03:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.548+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc198005440 con 0x559dd1325070 2024-09-21T03:03:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc198004d70 con 0x559dd1325070 2024-09-21T03:03:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 10 client.15702 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:01.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 10 client.15702 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 20 client.15702 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd175d790 0x559dd175fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 4 client.15702 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559dd1640850 con 0x559dd175d790 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.549+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd175d790 0x559dd175fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1917fa640 10 client.15702 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:01.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1ad87d640 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd175d790 0x559dd175fbb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bbdab69bb8ad6afa server_cookie=5fd2a6977e25172e in_seq=0 out_seq=0 2024-09-21T03:03:01.552 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1917fa640 1 -- 172.21.15.16:0/2589793610 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1510 (crc 0 0 0) 0x559dd1640850 con 0x559dd175d790 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1917fa640 10 client.15702 handle_command_reply: tid=0 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1b6055740 1 client.15702 shutdown 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.550+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 msgr2=0x559dd16dc610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fc198005940 tx=0x7fc198009260 comp rx=0 tx=0).stop 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 shutdown_connections 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x559dd175d790 0x559dd175fbb0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x559dd1325070 0x559dd16dc610 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1917fa640 0 client.15702 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559dd12b8da0 0x559dd16dcb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 --2- 172.21.15.16:0/2589793610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x559dd12fc1b0 0x559dd16dc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 >> 172.21.15.16:0/2589793610 conn(0x559dd1319460 msgr2=0x559dd16ce040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 shutdown_connections 2024-09-21T03:03:01.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.551+0000 7fc1b6055740 1 -- 172.21.15.16:0/2589793610 wait complete. 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.552+0000 7fc1b6055740 20 client.15702 trim_cache size 0 max 16384 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:03:01.554 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.555 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:03:01.556 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:01.557 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:01.558 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:01.558 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:01.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 msgr2=0x7fc1940b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 0x7fc1940b8c10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fc1a0005fd0 tx=0x7fc1a0009500 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 msgr2=0x7fc1a819c6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a819c6b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7fc1a400e9f0 tx=0x7fc1a400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 shutdown_connections 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc1940b6700 0x7fc1940b8c10 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc1a81094d0 0x7fc1a819cc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1a8109e20 0x7fc1a81a1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 --2- 172.21.15.16:0/3671538330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc1a8103b90 0x7fc1a819c6b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 >> 172.21.15.16:0/3671538330 conn(0x7fc1a80ff5e0 msgr2=0x7fc1a810e910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.553+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 shutdown_connections 2024-09-21T03:03:01.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:01.554+0000 7fc1afae1640 1 -- 172.21.15.16:0/3671538330 wait complete. 2024-09-21T03:03:02.647 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:03:02.647 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:03:03.099 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:03.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:02 smithi123 ceph-mon[34645]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 409 MiB/s wr, 177 op/s 2024-09-21T03:03:03.231 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:02 smithi016 ceph-mon[31253]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 409 MiB/s wr, 177 op/s 2024-09-21T03:03:03.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:02 smithi018 ceph-mon[34587]: pgmap v401: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 409 MiB/s wr, 177 op/s 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 -- 172.21.15.16:0/2032235627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 msgr2=0x7fcba0102bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 --2- 172.21.15.16:0/2032235627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba0102bb0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c009890 tx=0x7fcb9c02a380 comp rx=0 tx=0).stop 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 -- 172.21.15.16:0/2032235627 shutdown_connections 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 --2- 172.21.15.16:0/2032235627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba0107be0 0x7fcba01080c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 --2- 172.21.15.16:0/2032235627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba01030f0 0x7fcba0107620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 --2- 172.21.15.16:0/2032235627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba0102bb0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.015+0000 7fcba8454640 1 -- 172.21.15.16:0/2032235627 >> 172.21.15.16:0/2032235627 conn(0x7fcba0079730 msgr2=0x7fcba0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.016+0000 7fcba8454640 1 -- 172.21.15.16:0/2032235627 shutdown_connections 2024-09-21T03:03:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.016+0000 7fcba8454640 1 -- 172.21.15.16:0/2032235627 wait complete. 2024-09-21T03:03:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.017+0000 7fcba8454640 1 Processor -- start 2024-09-21T03:03:04.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.017+0000 7fcba8454640 1 -- start start 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba01a0b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 0x7fcba01a10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 0x7fcba01a6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcba011a090 con 0x7fcba01030f0 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcba0119f10 con 0x7fcba01027a0 2024-09-21T03:03:04.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba8454640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcba011a210 con 0x7fcba0107be0 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba69f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 0x7fcba01a6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba69f1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 0x7fcba01a6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51946/0 (socket says 172.21.15.16:51946) 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba69f1640 1 -- 172.21.15.16:0/3063005568 learned_addr learned my addr 172.21.15.16:0/3063005568 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 0x7fcba01a10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba61f0640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba01a0b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 -- 172.21.15.16:0/3063005568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 msgr2=0x7fcba01a6360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 0x7fcba01a6360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 -- 172.21.15.16:0/3063005568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 msgr2=0x7fcba01a0b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba01a0b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba59ef640 1 -- 172.21.15.16:0/3063005568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcba01a6a90 con 0x7fcba01030f0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.018+0000 7fcba61f0640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba01a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcba59ef640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 0x7fcba01a10c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcba006ad30 tx=0x7fcb9000ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb9000cc40 con 0x7fcba01030f0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcb9000cde0 con 0x7fcba01030f0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcb90018780 con 0x7fcba01030f0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcba01a6d80 con 0x7fcba01030f0 2024-09-21T03:03:04.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.019+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcba01ad4e0 con 0x7fcba01030f0 2024-09-21T03:03:04.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.021+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcba0107620 con 0x7fcba01030f0 2024-09-21T03:03:04.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.023+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcb90018920 con 0x7fcba01030f0 2024-09-21T03:03:04.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.024+0000 7fcb977fe640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 0x7fcb8c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.024+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fcb900e7950 con 0x7fcba01030f0 2024-09-21T03:03:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.024+0000 7fcba61f0640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 0x7fcb8c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:04.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.025+0000 7fcba61f0640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 0x7fcb8c0b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005940 tx=0x7fcb9c0058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:04.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.032+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcb900b22a0 con 0x7fcba01030f0 2024-09-21T03:03:04.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.327+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcba01ad710 con 0x7fcba01030f0 2024-09-21T03:03:04.332 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:03:04.332 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.329+0000 7fcb977fe640 1 -- 172.21.15.16:0/3063005568 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7fcb900b71e0 con 0x7fcba01030f0 2024-09-21T03:03:04.333 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:03:04.333 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:03:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.334+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 msgr2=0x7fcb8c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.334+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 0x7fcb8c0b8c10 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcb9c005940 tx=0x7fcb9c0058d0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.334+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 msgr2=0x7fcba01a10c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:04.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.334+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 0x7fcba01a10c0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fcba006ad30 tx=0x7fcb9000ee50 comp rx=0 tx=0).stop 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 shutdown_connections 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcb8c0b6700 0x7fcb8c0b8c10 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcba01030f0 0x7fcba01a10c0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcba0107be0 0x7fcba01a6360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 --2- 172.21.15.16:0/3063005568 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcba01027a0 0x7fcba01a0b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:04.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.337+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 >> 172.21.15.16:0/3063005568 conn(0x7fcba0079730 msgr2=0x7fcba01055f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:04.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.346+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 shutdown_connections 2024-09-21T03:03:04.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:04.346+0000 7fcba8454640 1 -- 172.21.15.16:0/3063005568 wait complete. 2024-09-21T03:03:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:04 smithi123 ceph-mon[34645]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 305 MiB/s wr, 109 op/s 2024-09-21T03:03:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3063005568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:04 smithi016 ceph-mon[31253]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 305 MiB/s wr, 109 op/s 2024-09-21T03:03:05.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3063005568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:05.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:04 smithi018 ceph-mon[34587]: pgmap v402: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 305 MiB/s wr, 109 op/s 2024-09-21T03:03:05.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3063005568' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:05.696 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:03:05.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 -- 172.21.15.16:0/275268396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109ec0 msgr2=0x7f5de81106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 --2- 172.21.15.16:0/275268396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109ec0 0x7f5de81106f0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009ca0 tx=0x7f5de002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 -- 172.21.15.16:0/275268396 shutdown_connections 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 --2- 172.21.15.16:0/275268396 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109ec0 0x7f5de81106f0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 --2- 172.21.15.16:0/275268396 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8109570 0x7f5de8109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 --2- 172.21.15.16:0/275268396 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8103c30 0x7f5de8104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 -- 172.21.15.16:0/275268396 >> 172.21.15.16:0/275268396 conn(0x7f5de807b8d0 msgr2=0x7f5de807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 -- 172.21.15.16:0/275268396 shutdown_connections 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.911+0000 7f5dede85640 1 -- 172.21.15.16:0/275268396 wait complete. 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 Processor -- start 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 -- start start 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 0x7f5de8075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 0x7f5de8076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8113ae0 con 0x7f5de8109570 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8113960 con 0x7f5de8103c30 2024-09-21T03:03:07.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5dede85640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5de8113c60 con 0x7f5de8109ec0 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41742/0 (socket says 172.21.15.16:41742) 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5ddffff640 1 -- 172.21.15.16:0/583392678 learned_addr learned my addr 172.21.15.16:0/583392678 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5de77fe640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 0x7f5de8075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.912+0000 7f5de7fff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 0x7f5de8076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 -- 172.21.15.16:0/583392678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 msgr2=0x7f5de8076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 0x7f5de8076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 -- 172.21.15.16:0/583392678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 msgr2=0x7f5de8075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 0x7f5de8075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 -- 172.21.15.16:0/583392678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de81a8730 con 0x7f5de8109570 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5de7fff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 0x7f5de8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5de77fe640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 0x7f5de8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5ddffff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e970 tx=0x7f5dd000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5de57fa640 1 -- 172.21.15.16:0/583392678 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd000cd80 con 0x7f5de8109570 2024-09-21T03:03:07.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5de57fa640 1 -- 172.21.15.16:0/583392678 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00035f0 con 0x7f5de8109570 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de819a1e0 con 0x7f5de8109570 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.913+0000 7f5de57fa640 1 -- 172.21.15.16:0/583392678 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd0010690 con 0x7f5de8109570 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.914+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de819a720 con 0x7f5de8109570 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.916+0000 7f5de57fa640 1 -- 172.21.15.16:0/583392678 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5dd0003aa0 con 0x7f5de8109570 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.916+0000 7f5df43f9740 1 Processor -- start 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.916+0000 7f5df43f9740 1 -- start start 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbe22c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbe2d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56497bb58ea0 con 0x56497b81a390 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56497bb482d0 con 0x56497b83b070 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5df43f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56497bb4c0e0 con 0x56497b8121b0 2024-09-21T03:03:07.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de57fa640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 0x7f5dd40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbe2d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40882/0 (socket says 172.21.15.16:40882) 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 -- 172.21.15.16:0/1134862739 learned_addr learned my addr 172.21.15.16:0/1134862739 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de7fff640 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbe22c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de7fff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 0x7f5dd40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de57fa640 1 -- 172.21.15.16:0/583392678 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5dd00e7a30 con 0x7f5de8109570 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 -- 172.21.15.16:0/1134862739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 msgr2=0x56497bbe22c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbe22c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 -- 172.21.15.16:0/1134862739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 msgr2=0x56497bbe2d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbe2d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.917+0000 7f5de77fe640 1 -- 172.21.15.16:0/1134862739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497b8a6650 con 0x56497b8121b0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.918+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009c50 tx=0x7f5dd8005f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15b821bd007dcf90 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.918+0000 7f5de7fff640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 0x7f5dd40b8b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005e30 tx=0x7f5de002a8c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.918+0000 7f5dddffb640 1 -- 172.21.15.16:0/1134862739 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd8003bd0 con 0x56497b8121b0 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.918+0000 7f5dddffb640 1 -- 172.21.15.16:0/1134862739 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8024070 con 0x56497b8121b0 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.918+0000 7f5dddffb640 1 -- 172.21.15.16:0/1134862739 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd801e380 con 0x56497b8121b0 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- 172.21.15.16:0/1134862739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 msgr2=0x56497bbe2800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8009c50 tx=0x7f5dd8005f90 comp rx=0 tx=0).stop 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- 172.21.15.16:0/1134862739 shutdown_connections 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbe2d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbe2800 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1134862739 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbe22c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- 172.21.15.16:0/1134862739 >> 172.21.15.16:0/1134862739 conn(0x56497b82f460 msgr2=0x56497bbe01b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- 172.21.15.16:0/1134862739 shutdown_connections 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- 172.21.15.16:0/1134862739 wait complete. 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.919+0000 7f5df43f9740 1 -- start start 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 Processor -- start 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 0x56497bbeddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56497b9a9700 con 0x56497b83b070 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56497bae3160 con 0x56497b8121b0 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56497bbf0aa0 con 0x56497b81a390 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5df43f9740 10 client.0 _ll_register_callbacks cb 0x56497bafc9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 0x56497bbeddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41774/0 (socket says 172.21.15.16:41774) 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5de7fff640 1 -- 172.21.15.16:0/1608003550 learned_addr learned my addr 172.21.15.16:0/1608003550 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5ddffff640 1 -- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 msgr2=0x56497bbed8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5ddffff640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbed8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.920+0000 7f5ddffff640 1 -- 172.21.15.16:0/1608003550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 msgr2=0x56497bbee330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5ddffff640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5ddffff640 1 -- 172.21.15.16:0/1608003550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497bb10c90 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5de7fff640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5ddffff640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 0x56497bbeddf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00e51a0 tx=0x7f5dd0079450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ae1883155e2ca47c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00ad1f0 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd8024070 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5dd0079020 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5dd00ad010 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5df43f9740 10 client.25360 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56497bbfc780 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd8024070 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.921+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5dd00aebd0 con 0x56497b81a390 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 10 client.25360 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:07.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 10 client.25360 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 20 client.25360 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7f99f0 0x56497bbffc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 4 client.25360 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56497bb5adf0 con 0x56497b7f99f0 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7f99f0 0x56497bbffc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5dc6ffd640 10 client.25360 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.922+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7f99f0 0x56497bbffc40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6c23374bc77b557d server_cookie=71a0579d6530a8d3 in_seq=0 out_seq=0 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1608003550 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56497bb5adf0 con 0x56497b7f99f0 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5dc6ffd640 10 client.25360 handle_command_reply: tid=0 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 client.25360 shutdown 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 msgr2=0x56497bbeddf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 0x56497bbeddf0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00e51a0 tx=0x7f5dd0079450 comp rx=0 tx=0).stop 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 shutdown_connections 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7f99f0 0x56497bbffc40 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b83b070 0x56497bbee330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b81a390 0x56497bbeddf0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5dc6ffd640 0 client.25360 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1608003550 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbed8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 >> 172.21.15.16:0/1608003550 conn(0x56497b82f460 msgr2=0x56497bbe5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 shutdown_connections 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 1 -- 172.21.15.16:0/1608003550 wait complete. 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.923+0000 7f5df43f9740 20 client.25360 trim_cache size 0 max 16384 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.936+0000 7f5df43f9740 1 Processor -- start 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.936+0000 7f5df43f9740 1 -- start start 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbef8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbeafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81ab420 con 0x56497b81a390 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81adca0 con 0x56497b83b070 2024-09-21T03:03:07.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5df43f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81aba90 con 0x56497b8121b0 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41780/0 (socket says 172.21.15.16:41780) 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 -- 172.21.15.16:0/4049848144 learned_addr learned my addr 172.21.15.16:0/4049848144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5de77fe640 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbef8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 -- 172.21.15.16:0/4049848144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 msgr2=0x56497bbef8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbef8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 -- 172.21.15.16:0/4049848144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 msgr2=0x56497bbeafe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5de7fff640 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbeafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbeafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 -- 172.21.15.16:0/4049848144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56497bbe83e0 con 0x56497b81a390 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.937+0000 7f5ddffff640 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098c0 tx=0x7f5dd00e7de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38fbb3ce45c562f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5ddd7fa640 1 -- 172.21.15.16:0/4049848144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00ad1f0 con 0x56497b81a390 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5ddd7fa640 1 -- 172.21.15.16:0/4049848144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00ad3b0 con 0x56497b81a390 2024-09-21T03:03:07.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5ddd7fa640 1 -- 172.21.15.16:0/4049848144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd00b2180 con 0x56497b81a390 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 -- 172.21.15.16:0/4049848144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 msgr2=0x56497bbeaaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f5dd00098c0 tx=0x7f5dd00e7de0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 -- 172.21.15.16:0/4049848144 shutdown_connections 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbeaaa0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b83b070 0x56497bbeafe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 --2- 172.21.15.16:0/4049848144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b8121b0 0x56497bbef8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 -- 172.21.15.16:0/4049848144 >> 172.21.15.16:0/4049848144 conn(0x56497b82f460 msgr2=0x56497bbe3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 -- 172.21.15.16:0/4049848144 shutdown_connections 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.938+0000 7f5df43f9740 1 -- 172.21.15.16:0/4049848144 wait complete. 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 -- start start 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 Processor -- start 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbf2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81ac160 con 0x56497b81a390 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81ad5d0 con 0x56497b8121b0 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5de81ac830 con 0x56497b83b070 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.939+0000 7f5df43f9740 10 client.0 _ll_register_callbacks cb 0x7f5dd00a9740 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5ddffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41794/0 (socket says 172.21.15.16:41794) 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40916/0 (socket says 172.21.15.16:40916) 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5ddffff640 1 -- 172.21.15.16:0/1333186745 learned_addr learned my addr 172.21.15.16:0/1333186745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbf2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 -- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 msgr2=0x56497bbf2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbf2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 -- 172.21.15.16:0/1333186745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 msgr2=0x56497bbf2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 -- 172.21.15.16:0/1333186745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56497bbf2e50 con 0x56497b83b070 2024-09-21T03:03:07.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5ddffff640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5de7fff640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009ca0 tx=0x7f5de00095c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=928eecc6807c66c1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5dc6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5de005b060 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5de0047070 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.940+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd00ad3b0 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5de0003e00 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 10 client.25366 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56497bc6ac00 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd00ad3b0 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5de004a040 con 0x56497b83b070 2024-09-21T03:03:07.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 10 client.25366 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 10 client.25366 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 20 client.25366 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7ceda0 0x56497bc6d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 4 client.25366 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56497bb568a0 con 0x56497b7ceda0 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.941+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7ceda0 0x56497bc6d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:07.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5de77fe640 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7ceda0 0x56497bc6d3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a41d1f7fd78c0ee2 server_cookie=850515018d3a2aec in_seq=0 out_seq=0 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5dc6ffd640 10 client.25366 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5dc6ffd640 1 -- 172.21.15.16:0/1333186745 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56497bb568a0 con 0x56497b7ceda0 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5dc6ffd640 10 client.25366 handle_command_reply: tid=0 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 client.25366 shutdown 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 msgr2=0x56497bc69640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5de0009ca0 tx=0x7f5de00095c0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 shutdown_connections 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56497b7ceda0 0x56497bc6d3f0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56497b81a390 0x56497bbf2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5dc6ffd640 0 client.25366 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56497b83b070 0x56497bc69640 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 --2- 172.21.15.16:0/1333186745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56497b8121b0 0x56497bbf2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 >> 172.21.15.16:0/1333186745 conn(0x56497b82f460 msgr2=0x56497bbe3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 shutdown_connections 2024-09-21T03:03:07.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 1 -- 172.21.15.16:0/1333186745 wait complete. 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.942+0000 7f5df43f9740 20 client.25366 trim_cache size 0 max 16384 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 msgr2=0x7f5dd40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 0x7f5dd40b8b90 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f5de0005e30 tx=0x7f5de002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 msgr2=0x7f5de8075c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f5dd000e970 tx=0x7f5dd000ee30 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 shutdown_connections 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5dd40b6680 0x7f5dd40b8b90 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5de8109570 0x7f5de8075c50 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5de8109ec0 0x7f5de8076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 --2- 172.21.15.16:0/583392678 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5de8103c30 0x7f5de8075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 >> 172.21.15.16:0/583392678 conn(0x7f5de807b8d0 msgr2=0x7f5de8105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 shutdown_connections 2024-09-21T03:03:07.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:06.943+0000 7f5dede85640 1 -- 172.21.15.16:0/583392678 wait complete. 2024-09-21T03:03:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:06 smithi123 ceph-mon[34645]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 421 MiB/s wr, 364 op/s 2024-09-21T03:03:07.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:06 smithi018 ceph-mon[34587]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 421 MiB/s wr, 364 op/s 2024-09-21T03:03:07.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:06 smithi016 ceph-mon[31253]: pgmap v403: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 421 MiB/s wr, 364 op/s 2024-09-21T03:03:09.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:08 smithi016 ceph-mon[31253]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 374 MiB/s wr, 531 op/s 2024-09-21T03:03:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:08 smithi018 ceph-mon[34587]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 374 MiB/s wr, 531 op/s 2024-09-21T03:03:09.443 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:03:09.443 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:03:09.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:08 smithi123 ceph-mon[34645]: pgmap v404: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1012 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 374 MiB/s wr, 531 op/s 2024-09-21T03:03:09.834 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:10.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:10 smithi016 ceph-mon[31253]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 381 MiB/s wr, 701 op/s 2024-09-21T03:03:10.373 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:10 smithi123 ceph-mon[34645]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 381 MiB/s wr, 701 op/s 2024-09-21T03:03:10.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:10 smithi018 ceph-mon[34587]: pgmap v405: 97 pgs: 97 active+clean; 20 GiB data, 61 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 381 MiB/s wr, 701 op/s 2024-09-21T03:03:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.732+0000 7f8bbeecb640 1 -- 172.21.15.16:0/4003449459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb81094b0 msgr2=0x7f8bb81098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.732+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/4003449459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb81094b0 0x7f8bb81098c0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009900 tx=0x7f8ba402a410 comp rx=0 tx=0).stop 2024-09-21T03:03:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 -- 172.21.15.16:0/4003449459 shutdown_connections 2024-09-21T03:03:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/4003449459 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb81094b0 0x7f8bb81098c0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/4003449459 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8109e00 0x7f8bb8110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/4003449459 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb8101f70 0x7f8bb8102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 -- 172.21.15.16:0/4003449459 >> 172.21.15.16:0/4003449459 conn(0x7f8bb8077c60 msgr2=0x7f8bb80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 -- 172.21.15.16:0/4003449459 shutdown_connections 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.733+0000 7f8bbeecb640 1 -- 172.21.15.16:0/4003449459 wait complete. 2024-09-21T03:03:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.734+0000 7f8bbeecb640 1 Processor -- start 2024-09-21T03:03:10.736 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.734+0000 7f8bbeecb640 1 -- start start 2024-09-21T03:03:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101f70 0x7f8bb819a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb81094b0 0x7f8bb819aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8113ac0 con 0x7f8bb8109e00 2024-09-21T03:03:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41818/0 (socket says 172.21.15.16:41818) 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 -- 172.21.15.16:0/626984917 learned_addr learned my addr 172.21.15.16:0/626984917 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8113940 con 0x7f8bb81094b0 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb8113c40 con 0x7f8bb8101f70 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101f70 0x7f8bb819a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb81094b0 0x7f8bb819aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 -- 172.21.15.16:0/626984917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101f70 msgr2=0x7f8bb819a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101f70 0x7f8bb819a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 -- 172.21.15.16:0/626984917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb81094b0 msgr2=0x7f8bb819aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb81094b0 0x7f8bb819aa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.735+0000 7f8bbd468640 1 -- 172.21.15.16:0/626984917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8bb81a0420 con 0x7f8bb8109e00 2024-09-21T03:03:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.736+0000 7f8bbd468640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009fd0 tx=0x7f8bb000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.736+0000 7f8bb5ffb640 1 -- 172.21.15.16:0/626984917 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0017070 con 0x7f8bb8109e00 2024-09-21T03:03:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.736+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8bb81a0710 con 0x7f8bb8109e00 2024-09-21T03:03:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.736+0000 7f8bb5ffb640 1 -- 172.21.15.16:0/626984917 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bb0003e20 con 0x7f8bb8109e00 2024-09-21T03:03:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.736+0000 7f8bb5ffb640 1 -- 172.21.15.16:0/626984917 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0010460 con 0x7f8bb8109e00 2024-09-21T03:03:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.737+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8bb81a6e70 con 0x7f8bb8109e00 2024-09-21T03:03:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.740+0000 7f8bc543f740 1 Processor -- start 2024-09-21T03:03:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bb5ffb640 1 -- 172.21.15.16:0/626984917 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8bb0003970 con 0x7f8bb8109e00 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 -- start start 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f7516490 0x55d8f78e6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f78e6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d8f785ce60 con 0x55d8f751e670 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d8f784c290 con 0x55d8f7516490 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bc543f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8f78500a0 con 0x55d8f753f350 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40956/0 (socket says 172.21.15.16:40956) 2024-09-21T03:03:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.741+0000 7f8bb7fff640 1 -- 172.21.15.16:0/2029648483 learned_addr learned my addr 172.21.15.16:0/2029648483 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f78e6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f7516490 0x55d8f78e6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 -- 172.21.15.16:0/2029648483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f7516490 msgr2=0x55d8f78e6970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f7516490 0x55d8f78e6970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 -- 172.21.15.16:0/2029648483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 msgr2=0x55d8f78e6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f78e6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 -- 172.21.15.16:0/2029648483 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f75aa930 con 0x55d8f753f350 2024-09-21T03:03:10.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009900 tx=0x7f8ba4031400 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c3e0402b5fabe6d6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.742+0000 7f8bb5ffb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 0x7f8ba80b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.743+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 0x7f8ba80b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.743+0000 7f8ba2ffd640 1 -- 172.21.15.16:0/2029648483 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4035040 con 0x55d8f753f350 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.743+0000 7f8ba2ffd640 1 -- 172.21.15.16:0/2029648483 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba4043020 con 0x55d8f753f350 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.743+0000 7f8ba2ffd640 1 -- 172.21.15.16:0/2029648483 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4003850 con 0x55d8f753f350 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.743+0000 7f8bb5ffb640 1 -- 172.21.15.16:0/626984917 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8bb00e7360 con 0x7f8bb8109e00 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- 172.21.15.16:0/2029648483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 msgr2=0x55d8f78e6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ba4009900 tx=0x7f8ba4031400 comp rx=0 tx=0).stop 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 0x7f8ba80b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55d8f78e50b0 tx=0x7f8bac00b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- 172.21.15.16:0/2029648483 shutdown_connections 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f78e6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f7516490 0x55d8f78e6970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 --2- 172.21.15.16:0/2029648483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f753f350 0x55d8f78e6430 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- 172.21.15.16:0/2029648483 >> 172.21.15.16:0/2029648483 conn(0x55d8f7533740 msgr2=0x55d8f78e4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- 172.21.15.16:0/2029648483 shutdown_connections 2024-09-21T03:03:10.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- 172.21.15.16:0/2029648483 wait complete. 2024-09-21T03:03:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.744+0000 7f8bc543f740 1 -- start start 2024-09-21T03:03:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 Processor -- start 2024-09-21T03:03:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78f2070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78fdf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d8f76ad890 con 0x55d8f7516490 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d8f77e6d40 con 0x55d8f753f350 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d8f78f4bb0 con 0x55d8f751e670 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bc543f740 10 client.0 _ll_register_callbacks cb 0x55d8f784ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bbcc67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78f2070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:40972/0 (socket says 172.21.15.16:40972) 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bb7fff640 1 -- 172.21.15.16:0/301024875 learned_addr learned my addr 172.21.15.16:0/301024875 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.745+0000 7f8bbd468640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78fdf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 -- 172.21.15.16:0/301024875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 msgr2=0x55d8f78f2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 -- 172.21.15.16:0/301024875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 msgr2=0x55d8f78f2070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78f2070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 -- 172.21.15.16:0/301024875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f78fe570 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bbd468640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78fdf00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00e4ad0 tx=0x7f8bb00e4fb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f0ecb016c29caa0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bb0020070 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8bb00a8b20 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.746+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba4043020 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8bb0003c60 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 10 client.25433 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8f7900e80 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba4043020 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f8bb00a8d20 con 0x55d8f753f350 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 10 client.25433 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 10 client.25433 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:10.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 20 client.25433 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74fdcd0 0x55d8f79042e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 4 client.25433 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.747+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8f785edb0 con 0x55d8f74fdcd0 2024-09-21T03:03:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.748+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74fdcd0 0x55d8f79042e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.748+0000 7f8b97fff640 10 client.25433 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.748+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74fdcd0 0x55d8f79042e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c401ce1314fca409 server_cookie=7683dde23caaccc9 in_seq=0 out_seq=0 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.749+0000 7f8b97fff640 1 -- 172.21.15.16:0/301024875 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d8f785edb0 con 0x55d8f74fdcd0 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.749+0000 7f8b97fff640 10 client.25433 handle_command_reply: tid=0 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.749+0000 7f8bc543f740 1 client.25433 shutdown 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 msgr2=0x55d8f78fdf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78fdf00 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8bb00e4ad0 tx=0x7f8bb00e4fb0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 shutdown_connections 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74fdcd0 0x55d8f79042e0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78fdf00 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 --2- 172.21.15.16:0/301024875 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78f2070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8b97fff640 0 client.25433 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 >> 172.21.15.16:0/301024875 conn(0x55d8f7533740 msgr2=0x55d8f78e95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 shutdown_connections 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.750+0000 7f8bc543f740 1 -- 172.21.15.16:0/301024875 wait complete. 2024-09-21T03:03:10.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.751+0000 7f8bc543f740 20 client.25433 trim_cache size 0 max 16384 2024-09-21T03:03:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.771+0000 7f8bc543f740 1 Processor -- start 2024-09-21T03:03:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.771+0000 7f8bc543f740 1 -- start start 2024-09-21T03:03:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f7516490 0x55d8f7909ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78eb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ad670 con 0x55d8f751e670 2024-09-21T03:03:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81afef0 con 0x55d8f753f350 2024-09-21T03:03:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bc543f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81adce0 con 0x55d8f7516490 2024-09-21T03:03:10.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41844/0 (socket says 172.21.15.16:41844) 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 -- 172.21.15.16:0/1233202829 learned_addr learned my addr 172.21.15.16:0/1233202829 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 -- 172.21.15.16:0/1233202829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f7516490 msgr2=0x55d8f7909ee0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f7516490 0x55d8f7909ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 -- 172.21.15.16:0/1233202829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 msgr2=0x55d8f78eb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78eb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.772+0000 7f8bb7fff640 1 -- 172.21.15.16:0/1233202829 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8f78ed700 con 0x55d8f751e670 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.773+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40041d0 tx=0x7f8ba4004200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aaa893bd986b288 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.773+0000 7f8ba27fc640 1 -- 172.21.15.16:0/1233202829 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba404c020 con 0x55d8f751e670 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.773+0000 7f8ba27fc640 1 -- 172.21.15.16:0/1233202829 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba4043070 con 0x55d8f751e670 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.773+0000 7f8ba27fc640 1 -- 172.21.15.16:0/1233202829 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ba4035040 con 0x55d8f751e670 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 -- 172.21.15.16:0/1233202829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 msgr2=0x55d8f790a420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f8ba40041d0 tx=0x7f8ba4004200 comp rx=0 tx=0).stop 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 -- 172.21.15.16:0/1233202829 shutdown_connections 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f751e670 0x55d8f790a420 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78eb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 --2- 172.21.15.16:0/1233202829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f7516490 0x55d8f7909ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 -- 172.21.15.16:0/1233202829 >> 172.21.15.16:0/1233202829 conn(0x55d8f7533740 msgr2=0x55d8f78f06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 -- 172.21.15.16:0/1233202829 shutdown_connections 2024-09-21T03:03:10.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.774+0000 7f8bc543f740 1 -- 172.21.15.16:0/1233202829 wait complete. 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.775+0000 7f8bc543f740 1 -- start start 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.775+0000 7f8bc543f740 1 Processor -- start 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f6ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78f73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81ae3b0 con 0x55d8f7516490 2024-09-21T03:03:10.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81af820 con 0x55d8f753f350 2024-09-21T03:03:10.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8bb81aea80 con 0x55d8f751e670 2024-09-21T03:03:10.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bc543f740 10 client.0 _ll_register_callbacks cb 0x7f8bb0004380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:10.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bbcc67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bbcc67640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41850/0 (socket says 172.21.15.16:41850) 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.776+0000 7f8bbcc67640 1 -- 172.21.15.16:0/772872835 learned_addr learned my addr 172.21.15.16:0/772872835 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f6ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 -- 172.21.15.16:0/772872835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 msgr2=0x55d8f78f6ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f6ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 -- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 msgr2=0x55d8f78f73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78f73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 -- 172.21.15.16:0/772872835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8f78e8cc0 con 0x55d8f7516490 2024-09-21T03:03:10.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8bbcc67640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bac01e630 tx=0x7f8bac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a6456edd97e3db4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8b97fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.777+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bac01edf0 con 0x55d8f7516490 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8bac005020 con 0x55d8f7516490 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ba4043070 con 0x55d8f7516490 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8bac038050 con 0x55d8f7516490 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8bc543f740 10 client.15744 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:10.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d8f7973d40 con 0x55d8f7516490 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ba4043070 con 0x55d8f7516490 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.778+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f8bac02a470 con 0x55d8f7516490 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 10 client.15744 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 10 client.15744 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 20 client.15744 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74d3080 0x55d8f79764e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 4 client.15744 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.779+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d8f785a860 con 0x55d8f74d3080 2024-09-21T03:03:10.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.780+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74d3080 0x55d8f79764e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:10.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.783+0000 7f8b97fff640 10 client.15744 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:10.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.783+0000 7f8bb7fff640 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74d3080 0x55d8f79764e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e2f278392d9fa667 server_cookie=671cea179a27634e in_seq=0 out_seq=0 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8b97fff640 1 -- 172.21.15.16:0/772872835 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d8f785a860 con 0x55d8f74d3080 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8b97fff640 10 client.15744 handle_command_reply: tid=0 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8bc543f740 1 client.15744 shutdown 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 msgr2=0x55d8f78fa820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f8bac01e630 tx=0x7f8bac016000 comp rx=0 tx=0).stop 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 shutdown_connections 2024-09-21T03:03:10.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.786+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d8f74d3080 0x55d8f79764e0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d8f753f350 0x55d8f78f73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8b97fff640 0 client.15744 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d8f751e670 0x55d8f78f6ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8bc543f740 1 --2- 172.21.15.16:0/772872835 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d8f7516490 0x55d8f78fa820 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 >> 172.21.15.16:0/772872835 conn(0x55d8f7533740 msgr2=0x55d8f78e9290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.787+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 shutdown_connections 2024-09-21T03:03:10.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.788+0000 7f8bc543f740 1 -- 172.21.15.16:0/772872835 wait complete. 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.788+0000 7f8bc543f740 20 client.15744 trim_cache size 0 max 16384 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "dd86dc76-816b-45f1-90ea-fc274a18fddc", 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:03:10.790 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 msgr2=0x7f8ba80b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 0x7f8ba80b8c10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x55d8f78e50b0 tx=0x7f8bac00b410 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 msgr2=0x7f8bb819fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f8bb0009fd0 tx=0x7f8bb000b260 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 shutdown_connections 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f8ba80b6700 0x7f8ba80b8c10 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8bb8109e00 0x7f8bb819fcf0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8bb81094b0 0x7f8bb819aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 --2- 172.21.15.16:0/626984917 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb8101f70 0x7f8bb819a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:10.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.789+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 >> 172.21.15.16:0/626984917 conn(0x7f8bb8077c60 msgr2=0x7f8bb8105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.790+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 shutdown_connections 2024-09-21T03:03:10.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:10.790+0000 7f8bbeecb640 1 -- 172.21.15.16:0/626984917 wait complete. 2024-09-21T03:03:11.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:11 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:03:11.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:11 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:11.388 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:11 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:03:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:11 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:03:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:11 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:11 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:03:11.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:11 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:03:11.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:11 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:11.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:11 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:03:11.864 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:03:12.309 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:12 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:12.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:12 smithi018 ceph-mon[34587]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 MiB/s wr, 1.28k op/s 2024-09-21T03:03:12.466 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:12 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:12.466 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:12 smithi016 ceph-mon[31253]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 MiB/s wr, 1.28k op/s 2024-09-21T03:03:12.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:12 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:12.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:12 smithi123 ceph-mon[34645]: pgmap v406: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 478 MiB/s wr, 1.28k op/s 2024-09-21T03:03:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.186+0000 7fb953215640 1 -- 172.21.15.16:0/2558681563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 msgr2=0x7fb94c1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.186+0000 7fb953215640 1 --2- 172.21.15.16:0/2558681563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c1087b0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb940009900 tx=0x7fb94002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 -- 172.21.15.16:0/2558681563 shutdown_connections 2024-09-21T03:03:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 --2- 172.21.15.16:0/2558681563 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c108cf0 0x7fb94c10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 --2- 172.21.15.16:0/2558681563 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c1087b0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 --2- 172.21.15.16:0/2558681563 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c102a60 0x7fb94c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 -- 172.21.15.16:0/2558681563 >> 172.21.15.16:0/2558681563 conn(0x7fb94c079730 msgr2=0x7fb94c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 -- 172.21.15.16:0/2558681563 shutdown_connections 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.202+0000 7fb953215640 1 -- 172.21.15.16:0/2558681563 wait complete. 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.205+0000 7fb953215640 1 Processor -- start 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.205+0000 7fb953215640 1 -- start start 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.206+0000 7fb953215640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 0x7fb94c199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.207+0000 7fb953215640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c199a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.207+0000 7fb953215640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 0x7fb94c19eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.210+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 0x7fb94c199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.210+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 0x7fb94c199460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41876/0 (socket says 172.21.15.16:41876) 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.210+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c199a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.210+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c199a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:38274/0 (socket says 172.21.15.16:38274) 2024-09-21T03:03:13.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.211+0000 7fb950fb1640 1 -- 172.21.15.16:0/631135920 learned_addr learned my addr 172.21.15.16:0/631135920 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.211+0000 7fb9517b2640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 0x7fb94c19eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.211+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1129b0 con 0x7fb94c102a60 2024-09-21T03:03:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.211+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c112830 con 0x7fb94c1083a0 2024-09-21T03:03:13.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.212+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c112b30 con 0x7fb94c108cf0 2024-09-21T03:03:13.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.214+0000 7fb9517b2640 1 -- 172.21.15.16:0/631135920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 msgr2=0x7fb94c199a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.214+0000 7fb9517b2640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c199a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.215+0000 7fb9517b2640 1 -- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 msgr2=0x7fb94c199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.215+0000 7fb9517b2640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 0x7fb94c199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.215+0000 7fb9517b2640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c19f310 con 0x7fb94c108cf0 2024-09-21T03:03:13.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.217+0000 7fb9517b2640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 0x7fb94c19eca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb94400cab0 tx=0x7fb94400cf70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.219+0000 7fb949ffb640 1 -- 172.21.15.16:0/631135920 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb94400ae20 con 0x7fb94c108cf0 2024-09-21T03:03:13.220 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.219+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c19f600 con 0x7fb94c108cf0 2024-09-21T03:03:13.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.220+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c19fab0 con 0x7fb94c108cf0 2024-09-21T03:03:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.222+0000 7fb949ffb640 1 -- 172.21.15.16:0/631135920 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb94400f900 con 0x7fb94c108cf0 2024-09-21T03:03:13.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.222+0000 7fb949ffb640 1 -- 172.21.15.16:0/631135920 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440189f0 con 0x7fb94c108cf0 2024-09-21T03:03:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.230+0000 7fb959789740 1 Processor -- start 2024-09-21T03:03:13.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.230+0000 7fb959789740 1 -- start start 2024-09-21T03:03:13.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.231+0000 7fb959789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0585650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.232+0000 7fb959789740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.233+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0585650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.233+0000 7fb959789740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b05860d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.233+0000 7fb959789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592b04fbe50 con 0x5592b01de070 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.233+0000 7fb959789740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592b04eb280 con 0x5592b01bd390 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.233+0000 7fb959789740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592b04ef090 con 0x5592b01b51b0 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.234+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.234+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41024/0 (socket says 172.21.15.16:41024) 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.234+0000 7fb950fb1640 1 -- 172.21.15.16:0/1812609814 learned_addr learned my addr 172.21.15.16:0/1812609814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb949ffb640 1 -- 172.21.15.16:0/631135920 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb94400fab0 con 0x7fb94c108cf0 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb950fb1640 1 -- 172.21.15.16:0/1812609814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 msgr2=0x5592b05860d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b05860d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb950fb1640 1 -- 172.21.15.16:0/1812609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 msgr2=0x5592b0585650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0585650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.235+0000 7fb950fb1640 1 -- 172.21.15.16:0/1812609814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b0249650 con 0x5592b01b51b0 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.237+0000 7fb949ffb640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 0x7fb93c0b8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.238+0000 7fb949ffb640 1 -- 172.21.15.16:0/631135920 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb9440e7950 con 0x7fb94c108cf0 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.238+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb938017fb0 tx=0x7fb938009df0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a7b212fd1933ad56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.240+0000 7fb936ffd640 1 -- 172.21.15.16:0/1812609814 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93800ec50 con 0x5592b01b51b0 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.240+0000 7fb936ffd640 1 -- 172.21.15.16:0/1812609814 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb93800edf0 con 0x5592b01b51b0 2024-09-21T03:03:13.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.240+0000 7fb936ffd640 1 -- 172.21.15.16:0/1812609814 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9380047a0 con 0x5592b01b51b0 2024-09-21T03:03:13.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.241+0000 7fb94bfff640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 0x7fb93c0b8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.242+0000 7fb959789740 1 -- 172.21.15.16:0/1812609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 msgr2=0x5592b0585b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.242+0000 7fb959789740 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fb938017fb0 tx=0x7fb938009df0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.242+0000 7fb94bfff640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 0x7fb93c0b8dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb94002a980 tx=0x7fb94002a910 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.243+0000 7fb959789740 1 -- 172.21.15.16:0/1812609814 shutdown_connections 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.243+0000 7fb959789740 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b05860d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.243+0000 7fb959789740 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b0585b90 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.243+0000 7fb959789740 1 --2- 172.21.15.16:0/1812609814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0585650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.243+0000 7fb959789740 1 -- 172.21.15.16:0/1812609814 >> 172.21.15.16:0/1812609814 conn(0x5592b01d2460 msgr2=0x5592b0583540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.244+0000 7fb959789740 1 -- 172.21.15.16:0/1812609814 shutdown_connections 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.245+0000 7fb959789740 1 -- 172.21.15.16:0/1812609814 wait complete. 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.246+0000 7fb959789740 1 -- start start 2024-09-21T03:03:13.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.246+0000 7fb959789740 1 Processor -- start 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb959789740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b058f3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb959789740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b058f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb959789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb959789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5592b034c700 con 0x5592b01de070 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb9517b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb9517b2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.248+0000 7fb9517b2640 1 -- 172.21.15.16:0/2121106795 learned_addr learned my addr 172.21.15.16:0/2121106795 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb950fb1640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b058f3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb94bfff640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b058f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5592b0486110 con 0x5592b01bd390 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5592b0593d80 con 0x5592b01b51b0 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb9517b2640 1 -- 172.21.15.16:0/2121106795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 msgr2=0x5592b058f3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb9517b2640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b058f3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb9517b2640 1 -- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 msgr2=0x5592b058f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb9517b2640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b058f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.250+0000 7fb9517b2640 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b04b3c40 con 0x5592b01de070 2024-09-21T03:03:13.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.250+0000 7fb9517b2640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9440ad6f0 tx=0x7fb9440e55e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=207257b563ef9de6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.249+0000 7fb959789740 10 client.0 _ll_register_callbacks cb 0x5592b049f990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:13.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.250+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb93800edf0 con 0x5592b01de070 2024-09-21T03:03:13.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb92bfff640 10 client.15753 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb92bfff640 1 -- 172.21.15.16:0/2121106795 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440a7e60 con 0x5592b01de070 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb92bfff640 1 -- 172.21.15.16:0/2121106795 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb9440ac9c0 con 0x5592b01de070 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb92bfff640 1 -- 172.21.15.16:0/2121106795 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb9440ade60 con 0x5592b01de070 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb959789740 10 client.15753 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.254+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=20}) -- 0x5592b059fb30 con 0x5592b01de070 2024-09-21T03:03:13.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb92bfff640 1 -- 172.21.15.16:0/2121106795 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb9440a90b0 con 0x5592b01de070 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 10 client.15753 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 10 client.15753 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 20 client.15753 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b019c9f0 0x5592b05a2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 4 client.15753 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.255+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb93800edf0 con 0x5592b019c9f0 2024-09-21T03:03:13.257 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.256+0000 7fb950fb1640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b019c9f0 0x5592b05a2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.283 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.282+0000 7fb950fb1640 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b019c9f0 0x5592b05a2ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=748c1a2de1e6db17 server_cookie=b5f0e0725a7e406a in_seq=0 out_seq=0 2024-09-21T03:03:13.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.283+0000 7fb92bfff640 10 client.15753 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.285+0000 7fb92bfff640 1 -- 172.21.15.16:0/2121106795 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7fb93800edf0 con 0x5592b019c9f0 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.285+0000 7fb92bfff640 10 client.15753 handle_command_reply: tid=0 2024-09-21T03:03:13.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.285+0000 7fb959789740 1 client.15753 shutdown 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.285+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 msgr2=0x5592b0596ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7fb9440ad6f0 tx=0x7fb9440e55e0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 shutdown_connections 2024-09-21T03:03:13.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b019c9f0 0x5592b05a2ef0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01de070 0x5592b0596ca0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b058f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 --2- 172.21.15.16:0/2121106795 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01b51b0 0x5592b058f3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 >> 172.21.15.16:0/2121106795 conn(0x5592b01d2460 msgr2=0x5592b0588830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb92bfff640 0 client.15753 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 shutdown_connections 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.286+0000 7fb959789740 1 -- 172.21.15.16:0/2121106795 wait complete. 2024-09-21T03:03:13.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.287+0000 7fb959789740 20 client.15753 trim_cache size 0 max 16384 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.365+0000 7fb959789740 1 Processor -- start 2024-09-21T03:03:13.367 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.365+0000 7fb959789740 1 -- start start 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.366+0000 7fb959789740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 0x5592b0588e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.366+0000 7fb959789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 0x5592b0582e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.368 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.367+0000 7fb959789740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b05833c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.367+0000 7fb959789740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1ab500 con 0x5592b01bd390 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.367+0000 7fb959789740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1add80 con 0x5592b01b51b0 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.367+0000 7fb959789740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1abb70 con 0x5592b01de070 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.368+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 0x5592b0582e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.368+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 0x5592b0582e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41906/0 (socket says 172.21.15.16:41906) 2024-09-21T03:03:13.369 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.368+0000 7fb94bfff640 1 -- 172.21.15.16:0/1481755293 learned_addr learned my addr 172.21.15.16:0/1481755293 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.368+0000 7fb9517b2640 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b05833c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.368+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 0x5592b0588e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.369+0000 7fb950fb1640 1 -- 172.21.15.16:0/1481755293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 msgr2=0x5592b05833c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.369+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b05833c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.369+0000 7fb950fb1640 1 -- 172.21.15.16:0/1481755293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 msgr2=0x5592b0582e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.369+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 0x5592b0582e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.369+0000 7fb950fb1640 1 -- 172.21.15.16:0/1481755293 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5592b0590470 con 0x5592b01b51b0 2024-09-21T03:03:13.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.370+0000 7fb950fb1640 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 0x5592b0588e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb938017710 tx=0x7fb9380173b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fe0e3a54b7fa324e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.370+0000 7fb9367fc640 1 -- 172.21.15.16:0/1481755293 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938017940 con 0x5592b01b51b0 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.371+0000 7fb9367fc640 1 -- 172.21.15.16:0/1481755293 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb938005040 con 0x5592b01b51b0 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.371+0000 7fb9367fc640 1 -- 172.21.15.16:0/1481755293 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb938017ae0 con 0x5592b01b51b0 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.372+0000 7fb959789740 1 -- 172.21.15.16:0/1481755293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 msgr2=0x5592b0588e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.374 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 0x5592b0588e60 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb938017710 tx=0x7fb9380173b0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 -- 172.21.15.16:0/1481755293 shutdown_connections 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01bd390 0x5592b0582e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b05833c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 --2- 172.21.15.16:0/1481755293 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01b51b0 0x5592b0588e60 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 -- 172.21.15.16:0/1481755293 >> 172.21.15.16:0/1481755293 conn(0x5592b01d2460 msgr2=0x5592b0588750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 -- 172.21.15.16:0/1481755293 shutdown_connections 2024-09-21T03:03:13.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.373+0000 7fb959789740 1 -- 172.21.15.16:0/1481755293 wait complete. 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.374+0000 7fb959789740 1 -- start start 2024-09-21T03:03:13.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.375+0000 7fb959789740 1 Processor -- start 2024-09-21T03:03:13.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.376+0000 7fb959789740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 0x5592b0599870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.376+0000 7fb959789740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b0599dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.377+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 0x5592b0599870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.377+0000 7fb950fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 0x5592b0599870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41912/0 (socket says 172.21.15.16:41912) 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.378+0000 7fb94bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b0599dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.378+0000 7fb950fb1640 1 -- 172.21.15.16:0/3776546877 learned_addr learned my addr 172.21.15.16:0/3776546877 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:13.380 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.379+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b060c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.379+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1ac240 con 0x5592b01b51b0 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.380+0000 7fb9517b2640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b060c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.381 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.380+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1ad6b0 con 0x5592b01bd390 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.380+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c1ac910 con 0x5592b01de070 2024-09-21T03:03:13.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.381+0000 7fb959789740 10 client.0 _ll_register_callbacks cb 0x7fb944018b90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.382+0000 7fb9517b2640 1 -- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 msgr2=0x5592b0599dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.382+0000 7fb9517b2640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b0599dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.382+0000 7fb9517b2640 1 -- 172.21.15.16:0/3776546877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 msgr2=0x5592b0599870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.382+0000 7fb9517b2640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 0x5592b0599870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.382+0000 7fb9517b2640 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5592b060ce60 con 0x5592b01de070 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.387+0000 7fb9517b2640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b060c870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9440a9d30 tx=0x7fb944018490 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7d3d6988a0edb55 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:13.389 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.388+0000 7fb92bfff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.389+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9440a7e60 con 0x5592b01de070 2024-09-21T03:03:13.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.390+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb9440ac430 con 0x5592b01de070 2024-09-21T03:03:13.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.391+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb938005040 con 0x5592b01de070 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.393+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb9440d3020 con 0x5592b01de070 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.393+0000 7fb959789740 10 client.25399 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:13.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.394+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5592b060e090 con 0x5592b01de070 2024-09-21T03:03:13.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.394+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9440b7180 con 0x5592b01de070 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.396+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb9440b33d0 con 0x5592b01de070 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 10 client.25399 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 10 client.25399 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 20 client.25399 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b0171da0 0x5592b0610810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 4 client.25399 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:03:13.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.397+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb938005040 con 0x5592b0171da0 2024-09-21T03:03:13.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.399+0000 7fb950fb1640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b0171da0 0x5592b0610810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.403+0000 7fb92bfff640 10 client.25399 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:13.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.403+0000 7fb950fb1640 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b0171da0 0x5592b0610810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5897bcc7c1784b49 server_cookie=2663d3f8ae541a02 in_seq=0 out_seq=0 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.407+0000 7fb92bfff640 1 -- 172.21.15.16:0/3776546877 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1990 (crc 0 0 0) 0x7fb938005040 con 0x5592b0171da0 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.407+0000 7fb92bfff640 10 client.25399 handle_command_reply: tid=0 2024-09-21T03:03:13.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.408+0000 7fb959789740 1 client.25399 shutdown 2024-09-21T03:03:13.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.409+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 msgr2=0x5592b060c870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.409+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b060c870 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fb9440a9d30 tx=0x7fb944018490 comp rx=0 tx=0).stop 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.409+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 shutdown_connections 2024-09-21T03:03:13.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5592b0171da0 0x5592b0610810 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5592b01de070 0x5592b060c870 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5592b01bd390 0x5592b0599dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb92bfff640 0 client.25399 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb959789740 1 --2- 172.21.15.16:0/3776546877 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5592b01b51b0 0x5592b0599870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.410+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 >> 172.21.15.16:0/3776546877 conn(0x5592b01d2460 msgr2=0x5592b0586680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.415+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 shutdown_connections 2024-09-21T03:03:13.417 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.416+0000 7fb959789740 1 -- 172.21.15.16:0/3776546877 wait complete. 2024-09-21T03:03:13.418 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.417+0000 7fb959789740 20 client.25399 trim_cache size 0 max 16384 2024-09-21T03:03:13.419 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "dd86dc76-816b-45f1-90ea-fc274a18fddc", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:03:13.420 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:03:13.421 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.422 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:13.423 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:13.424 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 msgr2=0x7fb93c0b8dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 0x7fb93c0b8dd0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fb94002a980 tx=0x7fb94002a910 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 msgr2=0x7fb94c19eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 0x7fb94c19eca0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fb94400cab0 tx=0x7fb94400cf70 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 shutdown_connections 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb93c0b68c0 0x7fb93c0b8dd0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb94c108cf0 0x7fb94c19eca0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb94c1083a0 0x7fb94c199a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 --2- 172.21.15.16:0/631135920 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb94c102a60 0x7fb94c199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 >> 172.21.15.16:0/631135920 conn(0x7fb94c079730 msgr2=0x7fb94c104510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:13.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 shutdown_connections 2024-09-21T03:03:13.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:13.424+0000 7fb953215640 1 -- 172.21.15.16:0/631135920 wait complete. 2024-09-21T03:03:14.601 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:14 smithi016 ceph-mon[31253]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 693 KiB/s rd, 376 MiB/s wr, 1.28k op/s 2024-09-21T03:03:14.605 INFO:tasks.cephfs.filesystem:scrub status for tag:dd86dc76-816b-45f1-90ea-fc274a18fddc - {'path': '/', 'tag': 'dd86dc76-816b-45f1-90ea-fc274a18fddc', 'options': 'recursive,force'} 2024-09-21T03:03:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:14 smithi018 ceph-mon[34587]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 693 KiB/s rd, 376 MiB/s wr, 1.28k op/s 2024-09-21T03:03:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:14 smithi123 ceph-mon[34645]: pgmap v407: 97 pgs: 97 active+clean; 20 GiB data, 62 GiB used, 1011 GiB / 1.0 TiB avail; 693 KiB/s rd, 376 MiB/s wr, 1.28k op/s 2024-09-21T03:03:16.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:16 smithi018 ceph-mon[34587]: pgmap v408: 97 pgs: 97 active+clean; 22 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 521 MiB/s wr, 2.03k op/s 2024-09-21T03:03:16.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:16 smithi123 ceph-mon[34645]: pgmap v408: 97 pgs: 97 active+clean; 22 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 521 MiB/s wr, 2.03k op/s 2024-09-21T03:03:16.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:16 smithi016 ceph-mon[31253]: pgmap v408: 97 pgs: 97 active+clean; 22 GiB data, 65 GiB used, 1008 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 521 MiB/s wr, 2.03k op/s 2024-09-21T03:03:18.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:18 smithi018 ceph-mon[34587]: pgmap v409: 97 pgs: 97 active+clean; 22 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 574 KiB/s rd, 451 MiB/s wr, 2.09k op/s 2024-09-21T03:03:18.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:18 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:03:18.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:18 smithi123 ceph-mon[34645]: pgmap v409: 97 pgs: 97 active+clean; 22 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 574 KiB/s rd, 451 MiB/s wr, 2.09k op/s 2024-09-21T03:03:18.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:18 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:03:18.770 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:18 smithi016 ceph-mon[31253]: pgmap v409: 97 pgs: 97 active+clean; 22 GiB data, 67 GiB used, 1006 GiB / 1.0 TiB avail; 574 KiB/s rd, 451 MiB/s wr, 2.09k op/s 2024-09-21T03:03:18.771 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:18 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:03:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:20 smithi018 ceph-mon[34587]: pgmap v410: 97 pgs: 97 active+clean; 23 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 858 KiB/s rd, 460 MiB/s wr, 2.13k op/s 2024-09-21T03:03:20.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:20 smithi123 ceph-mon[34645]: pgmap v410: 97 pgs: 97 active+clean; 23 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 858 KiB/s rd, 460 MiB/s wr, 2.13k op/s 2024-09-21T03:03:20.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:20 smithi016 ceph-mon[31253]: pgmap v410: 97 pgs: 97 active+clean; 23 GiB data, 68 GiB used, 1005 GiB / 1.0 TiB avail; 858 KiB/s rd, 460 MiB/s wr, 2.13k op/s 2024-09-21T03:03:22.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:22 smithi018 ceph-mon[34587]: pgmap v411: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 857 KiB/s rd, 608 MiB/s wr, 2.91k op/s 2024-09-21T03:03:22.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:22 smithi123 ceph-mon[34645]: pgmap v411: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 857 KiB/s rd, 608 MiB/s wr, 2.91k op/s 2024-09-21T03:03:22.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:22 smithi016 ceph-mon[31253]: pgmap v411: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1000 GiB / 1.0 TiB avail; 857 KiB/s rd, 608 MiB/s wr, 2.91k op/s 2024-09-21T03:03:24.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:24 smithi018 ceph-mon[34587]: pgmap v412: 97 pgs: 97 active+clean; 23 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 684 KiB/s rd, 514 MiB/s wr, 2.52k op/s 2024-09-21T03:03:24.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:24 smithi123 ceph-mon[34645]: pgmap v412: 97 pgs: 97 active+clean; 23 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 684 KiB/s rd, 514 MiB/s wr, 2.52k op/s 2024-09-21T03:03:24.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:24 smithi016 ceph-mon[31253]: pgmap v412: 97 pgs: 97 active+clean; 23 GiB data, 73 GiB used, 1000 GiB / 1.0 TiB avail; 684 KiB/s rd, 514 MiB/s wr, 2.52k op/s 2024-09-21T03:03:25.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:25 smithi018 ceph-mon[34587]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:25.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:25 smithi123 ceph-mon[34645]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:25.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:25 smithi016 ceph-mon[31253]: Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:26.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:26 smithi123 ceph-mon[34645]: pgmap v413: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 892 KiB/s rd, 549 MiB/s wr, 2.84k op/s 2024-09-21T03:03:26.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:26 smithi016 ceph-mon[31253]: pgmap v413: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 892 KiB/s rd, 549 MiB/s wr, 2.84k op/s 2024-09-21T03:03:26.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:26 smithi018 ceph-mon[34587]: pgmap v413: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 892 KiB/s rd, 549 MiB/s wr, 2.84k op/s 2024-09-21T03:03:28.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:28 smithi016 ceph-mon[31253]: pgmap v414: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 390 MiB/s wr, 2.10k op/s 2024-09-21T03:03:28.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:28 smithi018 ceph-mon[34587]: pgmap v414: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 390 MiB/s wr, 2.10k op/s 2024-09-21T03:03:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:28 smithi123 ceph-mon[34645]: pgmap v414: 97 pgs: 97 active+clean; 23 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 390 MiB/s wr, 2.10k op/s 2024-09-21T03:03:30.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:30 smithi016 ceph-mon[31253]: pgmap v415: 97 pgs: 97 active+clean; 24 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 354 MiB/s wr, 1.84k op/s 2024-09-21T03:03:30.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:30 smithi018 ceph-mon[34587]: pgmap v415: 97 pgs: 97 active+clean; 24 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 354 MiB/s wr, 1.84k op/s 2024-09-21T03:03:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:30 smithi123 ceph-mon[34645]: pgmap v415: 97 pgs: 97 active+clean; 24 GiB data, 72 GiB used, 1001 GiB / 1.0 TiB avail; 501 KiB/s rd, 354 MiB/s wr, 1.84k op/s 2024-09-21T03:03:31.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:31 smithi018 ceph-mon[34587]: Health check update: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:31.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:31 smithi123 ceph-mon[34645]: Health check update: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:32.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:31 smithi016 ceph-mon[31253]: Health check update: 2 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:32.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:32 smithi018 ceph-mon[34587]: pgmap v416: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 441 MiB/s wr, 1.88k op/s 2024-09-21T03:03:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:32 smithi123 ceph-mon[34645]: pgmap v416: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 441 MiB/s wr, 1.88k op/s 2024-09-21T03:03:33.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:32 smithi016 ceph-mon[31253]: pgmap v416: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 441 MiB/s wr, 1.88k op/s 2024-09-21T03:03:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:34 smithi018 ceph-mon[34587]: pgmap v417: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 287 MiB/s wr, 976 op/s 2024-09-21T03:03:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:34 smithi123 ceph-mon[34645]: pgmap v417: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 287 MiB/s wr, 976 op/s 2024-09-21T03:03:35.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:34 smithi016 ceph-mon[31253]: pgmap v417: 97 pgs: 97 active+clean; 24 GiB data, 74 GiB used, 999 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 287 MiB/s wr, 976 op/s 2024-09-21T03:03:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:36 smithi018 ceph-mon[34587]: pgmap v418: 97 pgs: 97 active+clean; 27 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 469 MiB/s wr, 1.53k op/s 2024-09-21T03:03:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:36 smithi018 ceph-mon[34587]: Health check update: 3 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:36 smithi123 ceph-mon[34645]: pgmap v418: 97 pgs: 97 active+clean; 27 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 469 MiB/s wr, 1.53k op/s 2024-09-21T03:03:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:36 smithi123 ceph-mon[34645]: Health check update: 3 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:37.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:36 smithi016 ceph-mon[31253]: pgmap v418: 97 pgs: 97 active+clean; 27 GiB data, 81 GiB used, 992 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 469 MiB/s wr, 1.53k op/s 2024-09-21T03:03:37.025 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:36 smithi016 ceph-mon[31253]: Health check update: 3 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:03:37.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:37 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:38.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:37 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:38.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:37 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:03:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:38 smithi123 ceph-mon[34645]: pgmap v419: 97 pgs: 97 active+clean; 28 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 490 MiB/s wr, 1.52k op/s 2024-09-21T03:03:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:03:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:38 smithi016 ceph-mon[31253]: pgmap v419: 97 pgs: 97 active+clean; 28 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 490 MiB/s wr, 1.52k op/s 2024-09-21T03:03:39.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:39.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:39.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:03:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:38 smithi018 ceph-mon[34587]: pgmap v419: 97 pgs: 97 active+clean; 28 GiB data, 83 GiB used, 990 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 490 MiB/s wr, 1.52k op/s 2024-09-21T03:03:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:03:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:03:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:03:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:40 smithi018 ceph-mon[34587]: pgmap v420: 97 pgs: 97 active+clean; 28 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 520 MiB/s wr, 1.70k op/s 2024-09-21T03:03:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:40 smithi123 ceph-mon[34645]: pgmap v420: 97 pgs: 97 active+clean; 28 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 520 MiB/s wr, 1.70k op/s 2024-09-21T03:03:41.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:40 smithi016 ceph-mon[31253]: pgmap v420: 97 pgs: 97 active+clean; 28 GiB data, 84 GiB used, 989 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 520 MiB/s wr, 1.70k op/s 2024-09-21T03:03:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:42 smithi018 ceph-mon[34587]: pgmap v421: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 644 MiB/s wr, 2.33k op/s 2024-09-21T03:03:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:42 smithi123 ceph-mon[34645]: pgmap v421: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 644 MiB/s wr, 2.33k op/s 2024-09-21T03:03:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:42 smithi016 ceph-mon[31253]: pgmap v421: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 644 MiB/s wr, 2.33k op/s 2024-09-21T03:03:44.606 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:03:44.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:44 smithi016 ceph-mon[31253]: pgmap v422: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 537 MiB/s wr, 2.12k op/s 2024-09-21T03:03:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:44 smithi018 ceph-mon[34587]: pgmap v422: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 537 MiB/s wr, 2.12k op/s 2024-09-21T03:03:45.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:44 smithi123 ceph-mon[34645]: pgmap v422: 97 pgs: 97 active+clean; 29 GiB data, 88 GiB used, 985 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 537 MiB/s wr, 2.12k op/s 2024-09-21T03:03:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/3997234643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8102a60 msgr2=0x7f4cc8102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.714 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/3997234643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8102a60 0x7f4cc8102e70 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8009900 tx=0x7f4cb802a410 comp rx=0 tx=0).stop 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/3997234643 shutdown_connections 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/3997234643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8108cf0 0x7f4cc810f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/3997234643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/3997234643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8102a60 0x7f4cc8102e70 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/3997234643 >> 172.21.15.16:0/3997234643 conn(0x7f4cc8079730 msgr2=0x7f4cc8079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/3997234643 shutdown_connections 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.618+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/3997234643 wait complete. 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.619+0000 7f4ccd8b7640 1 Processor -- start 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.619+0000 7f4ccd8b7640 1 -- start start 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81a10e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8108cf0 0x7f4cc81a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8112960 con 0x7f4cc8108cf0 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81127e0 con 0x7f4cc8102a60 2024-09-21T03:03:45.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4ccd8b7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc8112ae0 con 0x7f4cc81083a0 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4cc6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4cc6ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57398/0 (socket says 172.21.15.16:57398) 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/2250176140 learned_addr learned my addr 172.21.15.16:0/2250176140 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8108cf0 0x7f4cc81a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.620+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81a10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/2250176140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 msgr2=0x7f4cc81a10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81a10e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8108cf0 msgr2=0x7f4cc81a6380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8108cf0 0x7f4cc81a6380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/2250176140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cc81a6ab0 con 0x7f4cc8102a60 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:45.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005940 tx=0x7f4cb8004270 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4ccc8b5640 1 -- 172.21.15.16:0/2250176140 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb803f030 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4ccc8b5640 1 -- 172.21.15.16:0/2250176140 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb8003aa0 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cc81a6d40 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4ccc8b5640 1 -- 172.21.15.16:0/2250176140 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8035070 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.621+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cc81ad500 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.624+0000 7f4ccc8b5640 1 -- 172.21.15.16:0/2250176140 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4cb8042030 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.625+0000 7f4ccc8b5640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 0x7f4cb40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.625+0000 7f4ccc8b5640 1 -- 172.21.15.16:0/2250176140 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4cb8107b00 con 0x7f4cc8102a60 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.625+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 0x7f4cb40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.625+0000 7f4cd3e2b740 1 Processor -- start 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 -- start start 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 0x7f4cb40b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8100b70 tx=0x7f4cb000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf3948650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 0x55baf39490d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55baf38bee50 con 0x55baf35781b0 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55baf38ae280 con 0x55baf35a1070 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baf38b2090 con 0x55baf3580390 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46766/0 (socket says 172.21.15.16:46766) 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cbe7fc640 1 -- 172.21.15.16:0/1908295652 learned_addr learned my addr 172.21.15.16:0/1908295652 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf3948650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 0x55baf39490d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/1908295652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 msgr2=0x55baf39490d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 0x55baf39490d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/1908295652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 msgr2=0x55baf3948b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.626+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/1908295652 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baf360c650 con 0x55baf35a1070 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 0x55baf39490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:45.718 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf3948650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8105270 tx=0x7f4cb80ca0f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9ca307ba341717a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cbdffb640 1 -- 172.21.15.16:0/1908295652 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8049040 con 0x55baf35a1070 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cbdffb640 1 -- 172.21.15.16:0/1908295652 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb80cd530 con 0x55baf35a1070 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.627+0000 7f4cbdffb640 1 -- 172.21.15.16:0/1908295652 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80d6f00 con 0x55baf35a1070 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/1908295652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 msgr2=0x55baf3948650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf3948650 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8105270 tx=0x7f4cb80ca0f0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/1908295652 shutdown_connections 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf3948b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf3580390 0x55baf39490d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/1908295652 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf3948650 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/1908295652 >> 172.21.15.16:0/1908295652 conn(0x55baf3595460 msgr2=0x55baf3946540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/1908295652 shutdown_connections 2024-09-21T03:03:45.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.628+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/1908295652 wait complete. 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 -- start start 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 Processor -- start 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3953d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 0x55baf3954260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf395fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55baf370f700 con 0x55baf3580390 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55baf3849110 con 0x55baf35a1070 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.629+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55baf3956bf0 con 0x55baf35781b0 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cd3e2b740 10 client.0 _ll_register_callbacks cb 0x55baf3862990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 0x55baf3954260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 0x55baf3954260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46776/0 (socket says 172.21.15.16:46776) 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cbe7fc640 1 -- 172.21.15.16:0/3042528607 learned_addr learned my addr 172.21.15.16:0/3042528607 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3953d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf395fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 msgr2=0x55baf395fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf395fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/3042528607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 msgr2=0x55baf3954260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 0x55baf3954260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/3042528607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baf3876c40 con 0x55baf35781b0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3953d20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80cd720 tx=0x7f4cb81051e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8cb5e57c410525b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8049070 con 0x55baf35781b0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4cb80d7100 con 0x55baf35781b0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.630+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb80cd530 con 0x55baf35781b0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4cb80cd370 con 0x55baf35781b0 2024-09-21T03:03:45.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 10 client.25411 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55baf3962800 con 0x55baf35781b0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb80cd530 con 0x55baf35781b0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4cb80f2070 con 0x55baf35781b0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 10 client.25411 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 10 client.25411 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 20 client.25411 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf355f9f0 0x55baf3965b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 4 client.25411 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55baf38c0da0 con 0x55baf355f9f0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.631+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf355f9f0 0x55baf3965b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.632+0000 7f4ca6ffd640 10 client.25411 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.632+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf355f9f0 0x55baf3965b60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a3a1d17b1caea3 server_cookie=5166c6069f0beb79 in_seq=0 out_seq=0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/3042528607 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55baf38c0da0 con 0x55baf355f9f0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4ca6ffd640 10 client.25411 handle_command_reply: tid=0 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 client.25411 shutdown 2024-09-21T03:03:45.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 msgr2=0x55baf3953d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3953d20 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f4cb80cd720 tx=0x7f4cb81051e0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 shutdown_connections 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf355f9f0 0x55baf3965b60 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf3580390 0x55baf3954260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf35a1070 0x55baf395fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4ca6ffd640 0 client.25411 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/3042528607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3953d20 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.633+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 >> 172.21.15.16:0/3042528607 conn(0x55baf3595460 msgr2=0x55baf394b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.634+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 shutdown_connections 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.634+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/3042528607 wait complete. 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.634+0000 7f4cd3e2b740 20 client.25411 trim_cache size 0 max 16384 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.654+0000 7f4cd3e2b740 1 Processor -- start 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.654+0000 7f4cd3e2b740 1 -- start start 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf3951450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b3cb0 con 0x55baf355f9f0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b6530 con 0x55baf3580390 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b4320 con 0x55baf35781b0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf3951450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46778/0 (socket says 172.21.15.16:46778) 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34368/0 (socket says 172.21.15.16:34368) 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/98270571 learned_addr learned my addr 172.21.15.16:0/98270571 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/98270571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 msgr2=0x55baf3950f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/98270571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 msgr2=0x55baf3951450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf3951450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 -- 172.21.15.16:0/98270571 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55baf3965ed0 con 0x55baf355f9f0 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cbe7fc640 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:45.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf3951450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.655+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4cb803bb40 tx=0x7f4cb803b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=455631d6266c3794 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.656+0000 7f4cbd7fa640 1 -- 172.21.15.16:0/98270571 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb80e7070 con 0x55baf355f9f0 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.656+0000 7f4cbd7fa640 1 -- 172.21.15.16:0/98270571 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb80cd530 con 0x55baf355f9f0 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.656+0000 7f4cbd7fa640 1 -- 172.21.15.16:0/98270571 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cb8049070 con 0x55baf355f9f0 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/98270571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 msgr2=0x55baf395fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f4cb803bb40 tx=0x7f4cb803b460 comp rx=0 tx=0).stop 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/98270571 shutdown_connections 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf3951450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf35781b0 0x55baf3950f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/98270571 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf355f9f0 0x55baf395fd30 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/98270571 >> 172.21.15.16:0/98270571 conn(0x55baf3595460 msgr2=0x55baf39525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/98270571 shutdown_connections 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/98270571 wait complete. 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.657+0000 7f4cd3e2b740 1 -- start start 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 Processor -- start 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf355f9f0 0x55baf3958060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.725 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf39585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.726 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.727 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.728 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b49f0 con 0x55baf35781b0 2024-09-21T03:03:45.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b5e60 con 0x55baf3580390 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4cc81b50c0 con 0x55baf355f9f0 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cd3e2b740 10 client.0 _ll_register_callbacks cb 0x7f4cb80f1090 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cbe7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf39585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57440/0 (socket says 172.21.15.16:57440) 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf355f9f0 0x55baf3958060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 -- 172.21.15.16:0/4046439837 learned_addr learned my addr 172.21.15.16:0/4046439837 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 -- 172.21.15.16:0/4046439837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf355f9f0 msgr2=0x55baf3958060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf355f9f0 0x55baf3958060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 -- 172.21.15.16:0/4046439837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 msgr2=0x55baf39585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.658+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf39585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4cc77fe640 1 -- 172.21.15.16:0/4046439837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55baf3958e20 con 0x55baf3580390 2024-09-21T03:03:45.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4cc77fe640 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00042f0 tx=0x7f4cc0003f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afcd4401db257019 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4ca6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cc0022060 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4cc0012bb0 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cb80cd530 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4cc0012db0 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4cd3e2b740 10 client.25466 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.659+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55baf39d0950 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cb80cd530 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4cc0010430 con 0x55baf3580390 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 10 client.25466 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 10 client.25466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 20 client.25466 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf35a1070 0x55baf39d3170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 4 client.25466 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55baf38bc850 con 0x55baf35a1070 2024-09-21T03:03:45.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf35a1070 0x55baf39d3170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.660+0000 7f4ca6ffd640 10 client.25466 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cc6ffd640 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf35a1070 0x55baf39d3170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ac07722f89980de1 server_cookie=ac0f695d9f6560b in_seq=0 out_seq=0 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4ca6ffd640 1 -- 172.21.15.16:0/4046439837 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+1794 (crc 0 0 0) 0x55baf38bc850 con 0x55baf35a1070 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4ca6ffd640 10 client.25466 handle_command_reply: tid=0 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 client.25466 shutdown 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 msgr2=0x55baf39cf360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f4cc00042f0 tx=0x7f4cc0003f90 comp rx=0 tx=0).stop 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 shutdown_connections 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55baf35a1070 0x55baf39d3170 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55baf35781b0 0x55baf39585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55baf3580390 0x55baf39cf360 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4ca6ffd640 0 client.25466 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 --2- 172.21.15.16:0/4046439837 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55baf355f9f0 0x55baf3958060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 >> 172.21.15.16:0/4046439837 conn(0x55baf3595460 msgr2=0x55baf3949660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 shutdown_connections 2024-09-21T03:03:45.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 1 -- 172.21.15.16:0/4046439837 wait complete. 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.661+0000 7f4cd3e2b740 20 client.25466 trim_cache size 0 max 16384 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 msgr2=0x7f4cb40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 0x7f4cb40b8b90 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f4cc8100b70 tx=0x7f4cb000b390 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 msgr2=0x7f4cc81a0b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f4cb8005940 tx=0x7f4cb8004270 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 shutdown_connections 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4cb40b6680 0x7f4cb40b8b90 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4cc8108cf0 0x7f4cc81a6380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4cc81083a0 0x7f4cc81a10e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 --2- 172.21.15.16:0/2250176140 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4cc8102a60 0x7f4cc81a0b40 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 >> 172.21.15.16:0/2250176140 conn(0x7f4cc8079730 msgr2=0x7f4cc8101980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 shutdown_connections 2024-09-21T03:03:45.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:45.662+0000 7f4ccd8b7640 1 -- 172.21.15.16:0/2250176140 wait complete. 2024-09-21T03:03:46.647 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:03:46.648 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:03:47.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:46 smithi016 ceph-mon[31253]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 644 MiB/s wr, 2.12k op/s 2024-09-21T03:03:47.064 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:46 smithi018 ceph-mon[34587]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 644 MiB/s wr, 2.12k op/s 2024-09-21T03:03:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:46 smithi123 ceph-mon[34645]: pgmap v423: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 644 MiB/s wr, 2.12k op/s 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 -- 172.21.15.16:0/3085523220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7064109e20 msgr2=0x7f7064110650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 --2- 172.21.15.16:0/3085523220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7064109e20 0x7f7064110650 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f705c009ca0 tx=0x7f705c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 -- 172.21.15.16:0/3085523220 shutdown_connections 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 --2- 172.21.15.16:0/3085523220 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7064109e20 0x7f7064110650 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 --2- 172.21.15.16:0/3085523220 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f70641094d0 0x7f70641098e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 --2- 172.21.15.16:0/3085523220 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7064103b90 0x7f7064103fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 -- 172.21.15.16:0/3085523220 >> 172.21.15.16:0/3085523220 conn(0x7f70640ff5e0 msgr2=0x7f7064101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:47.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 -- 172.21.15.16:0/3085523220 shutdown_connections 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.737+0000 7f706bbd7640 1 -- 172.21.15.16:0/3085523220 wait complete. 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.738+0000 7f706bbd7640 1 Processor -- start 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.738+0000 7f706bbd7640 1 -- start start 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 0x7f706419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f7069973640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7064109e20 0x7f70641a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f7069973640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57458/0 (socket says 172.21.15.16:57458) 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113ae0 con 0x7f70641094d0 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113960 con 0x7f7064103b90 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f706bbd7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7064113c60 con 0x7f7064109e20 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.739+0000 7f7069973640 1 -- 172.21.15.16:0/450717916 learned_addr learned my addr 172.21.15.16:0/450717916 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069172640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 0x7f706419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:47.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 -- 172.21.15.16:0/450717916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7064109e20 msgr2=0x7f70641a1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7064109e20 0x7f70641a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 -- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 msgr2=0x7f706419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 0x7f706419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 -- 172.21.15.16:0/450717916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70641a2580 con 0x7f7064103b90 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069172640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 0x7f706419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f7069973640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f706000e9f0 tx=0x7f706000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f706000cd80 con 0x7f7064103b90 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70600108e0 con 0x7f7064103b90 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70641a2870 con 0x7f7064103b90 2024-09-21T03:03:47.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.740+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7060018980 con 0x7f7064103b90 2024-09-21T03:03:47.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.741+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70641a8fd0 con 0x7f7064103b90 2024-09-21T03:03:47.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.742+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7064103fa0 con 0x7f7064103b90 2024-09-21T03:03:47.746 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.744+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7060018b20 con 0x7f7064103b90 2024-09-21T03:03:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.745+0000 7f705affd640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 0x7f70500b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.746+0000 7f7069172640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 0x7f70500b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:47.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.746+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f70600e7c60 con 0x7f7064103b90 2024-09-21T03:03:47.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.746+0000 7f7069172640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 0x7f70500b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7064109c40 tx=0x7f7054005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:47.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:47.749+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7060056710 con 0x7f7064103b90 2024-09-21T03:03:48.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.123+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f706419d740 con 0x7f7064103b90 2024-09-21T03:03:48.126 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.124+0000 7f705affd640 1 -- 172.21.15.16:0/450717916 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f70600ae1d0 con 0x7f7064103b90 2024-09-21T03:03:48.126 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:03:48.127 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:03:48.127 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 msgr2=0x7f70500b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 0x7f70500b8c10 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f7064109c40 tx=0x7f7054005f90 comp rx=0 tx=0).stop 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 msgr2=0x7f706419c610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f706000e9f0 tx=0x7f706000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 shutdown_connections 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f70500b6700 0x7f70500b8c10 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f70641094d0 0x7f706419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7064109e20 0x7f70641a1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 --2- 172.21.15.16:0/450717916 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7064103b90 0x7f706419c610 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 >> 172.21.15.16:0/450717916 conn(0x7f70640ff5e0 msgr2=0x7f7064106aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 shutdown_connections 2024-09-21T03:03:48.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:48.126+0000 7f706bbd7640 1 -- 172.21.15.16:0/450717916 wait complete. 2024-09-21T03:03:49.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:48 smithi016 ceph-mon[31253]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 464 MiB/s wr, 1.40k op/s 2024-09-21T03:03:49.145 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:48 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/450717916' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:49.147 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:03:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:48 smithi018 ceph-mon[34587]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 464 MiB/s wr, 1.40k op/s 2024-09-21T03:03:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:48 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/450717916' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:48 smithi123 ceph-mon[34645]: pgmap v424: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 464 MiB/s wr, 1.40k op/s 2024-09-21T03:03:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:48 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/450717916' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:03:49.574 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 -- 172.21.15.16:0/3461862180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03001083d0 msgr2=0x7f03001087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 --2- 172.21.15.16:0/3461862180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03001083d0 0x7f03001087e0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f02f0009ca0 tx=0x7f02f002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 -- 172.21.15.16:0/3461862180 shutdown_connections 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 --2- 172.21.15.16:0/3461862180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03001083d0 0x7f03001087e0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 --2- 172.21.15.16:0/3461862180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0300108d20 0x7f030010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 --2- 172.21.15.16:0/3461862180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300102ae0 0x7f0300102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 -- 172.21.15.16:0/3461862180 >> 172.21.15.16:0/3461862180 conn(0x7f0300079790 msgr2=0x7f0300079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 -- 172.21.15.16:0/3461862180 shutdown_connections 2024-09-21T03:03:50.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.161+0000 7f0306acb640 1 -- 172.21.15.16:0/3461862180 wait complete. 2024-09-21T03:03:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 Processor -- start 2024-09-21T03:03:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 -- start start 2024-09-21T03:03:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03001083d0 0x7f030019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 0x7f030019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03001129e0 con 0x7f0300102ae0 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0300112860 con 0x7f0300108d20 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0306acb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0300112b60 con 0x7f03001083d0 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0304867640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0304867640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46820/0 (socket says 172.21.15.16:46820) 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0304867640 1 -- 172.21.15.16:0/1244725087 learned_addr learned my addr 172.21.15.16:0/1244725087 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f0305068640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 0x7f030019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.162+0000 7f02fffff640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03001083d0 0x7f030019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 -- 172.21.15.16:0/1244725087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03001083d0 msgr2=0x7f030019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03001083d0 0x7f030019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 -- 172.21.15.16:0/1244725087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 msgr2=0x7f030019fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 0x7f030019fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 -- 172.21.15.16:0/1244725087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f03001a0430 con 0x7f0300102ae0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0305068640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 0x7f030019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0304867640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02e800ad30 tx=0x7f02e8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f02fdffb640 1 -- 172.21.15.16:0/1244725087 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8004020 con 0x7f0300102ae0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f03001a0720 con 0x7f0300102ae0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.163+0000 7f02fdffb640 1 -- 172.21.15.16:0/1244725087 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e80054d0 con 0x7f0300102ae0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.164+0000 7f02fdffb640 1 -- 172.21.15.16:0/1244725087 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e8004b10 con 0x7f0300102ae0 2024-09-21T03:03:50.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.164+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03000ff550 con 0x7f0300102ae0 2024-09-21T03:03:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.166+0000 7f030d03f740 1 Processor -- start 2024-09-21T03:03:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.166+0000 7f02fdffb640 1 -- 172.21.15.16:0/1244725087 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f02e801f020 con 0x7f0300102ae0 2024-09-21T03:03:50.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.166+0000 7f030d03f740 1 -- start start 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 0x5618110616a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 0x561811061be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811062120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561810fd7ea0 con 0x561810cba070 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561810fc72d0 con 0x561810c911b0 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f030d03f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561810fcb0e0 con 0x561810c99390 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f0305068640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 0x5618110616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f0305068640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 0x5618110616a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46830/0 (socket says 172.21.15.16:46830) 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f0305068640 1 -- 172.21.15.16:0/2558127978 learned_addr learned my addr 172.21.15.16:0/2558127978 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f0304867640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 0x561811061be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.167+0000 7f02fffff640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811062120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02fdffb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 0x7f02ec0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 -- 172.21.15.16:0/2558127978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 msgr2=0x561811062120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811062120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 -- 172.21.15.16:0/2558127978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 msgr2=0x5618110616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 0x5618110616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 -- 172.21.15.16:0/2558127978 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561810d25650 con 0x561810c911b0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02fffff640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811062120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0305068640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 0x7f02ec0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f0304867640 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 0x561811061be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02e80e82f0 tx=0x7f02e80e1490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54ea0c1ccfdbb515 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02fdffb640 1 -- 172.21.15.16:0/1244725087 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f02e80eab80 con 0x7f0300102ae0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02f6ffd640 1 -- 172.21.15.16:0/2558127978 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80ee020 con 0x561810c911b0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02f6ffd640 1 -- 172.21.15.16:0/2558127978 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e80b0500 con 0x561810c911b0 2024-09-21T03:03:50.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.168+0000 7f02f6ffd640 1 -- 172.21.15.16:0/2558127978 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e801c030 con 0x561810c911b0 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 -- 172.21.15.16:0/2558127978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 msgr2=0x561811061be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f0305068640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 0x7f02ec0b8c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fd0 tx=0x7f02f80046c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 0x561811061be0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f02e80e82f0 tx=0x7f02e80e1490 comp rx=0 tx=0).stop 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 -- 172.21.15.16:0/2558127978 shutdown_connections 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811062120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c911b0 0x561811061be0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 --2- 172.21.15.16:0/2558127978 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810cba070 0x5618110616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 -- 172.21.15.16:0/2558127978 >> 172.21.15.16:0/2558127978 conn(0x561810cae460 msgr2=0x56181105f590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 -- 172.21.15.16:0/2558127978 shutdown_connections 2024-09-21T03:03:50.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.169+0000 7f030d03f740 1 -- 172.21.15.16:0/2558127978 wait complete. 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 -- start start 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 Processor -- start 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x56181106cdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x56181106d310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561810e28700 con 0x561810c911b0 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561810f62160 con 0x561810cba070 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.170+0000 7f030d03f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56181106fdc0 con 0x561810c99390 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f030d03f740 10 client.0 _ll_register_callbacks cb 0x561810f7b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57474/0 (socket says 172.21.15.16:57474) 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 -- 172.21.15.16:0/702435626 learned_addr learned my addr 172.21.15.16:0/702435626 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f02fffff640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x56181106d310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0304867640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x56181106cdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 -- 172.21.15.16:0/702435626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 msgr2=0x56181106d310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x56181106d310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 -- 172.21.15.16:0/702435626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 msgr2=0x56181106cdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x56181106cdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 -- 172.21.15.16:0/702435626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561810f8fc90 con 0x561810cba070 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0304867640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x56181106cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f02fffff640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x56181106d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f0305068640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02f800fe60 tx=0x7f02f8004730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a14a589faad9ef3d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.171+0000 7f02dffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f802b060 con 0x561810cba070 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f02f801b7c0 con 0x561810cba070 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02e80b0500 con 0x561810cba070 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f02f801b9c0 con 0x561810cba070 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 10 client.25481 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:50.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56181107bb40 con 0x561810cba070 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02e80b0500 con 0x561810cba070 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f02f8025030 con 0x561810cba070 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 10 client.25481 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 10 client.25481 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 20 client.25481 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810c789f0 0x56181107f060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 4 client.25481 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.172+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561810fd9df0 con 0x561810c789f0 2024-09-21T03:03:50.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.173+0000 7f0304867640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810c789f0 0x56181107f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.173+0000 7f02dffff640 10 client.25481 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.173+0000 7f0304867640 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810c789f0 0x56181107f060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44db7ca4383b40a2 server_cookie=80f3cae5074e2503 in_seq=0 out_seq=0 2024-09-21T03:03:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f02dffff640 1 -- 172.21.15.16:0/702435626 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561810fd9df0 con 0x561810c789f0 2024-09-21T03:03:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f02dffff640 10 client.25481 handle_command_reply: tid=0 2024-09-21T03:03:50.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 client.25481 shutdown 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 msgr2=0x5618110790b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f02f800fe60 tx=0x7f02f8004730 comp rx=0 tx=0).stop 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 shutdown_connections 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810c789f0 0x56181107f060 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cba070 0x5618110790b0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x56181106d310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 --2- 172.21.15.16:0/702435626 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x56181106cdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f02dffff640 0 client.25481 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.174+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 >> 172.21.15.16:0/702435626 conn(0x561810cae460 msgr2=0x561811064880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.175+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 shutdown_connections 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.175+0000 7f030d03f740 1 -- 172.21.15.16:0/702435626 wait complete. 2024-09-21T03:03:50.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.175+0000 7f030d03f740 20 client.25481 trim_cache size 0 max 16384 2024-09-21T03:03:50.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 Processor -- start 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 -- start start 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 0x561811064eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811069fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ab5a0 con 0x561810c911b0 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ade20 con 0x561810cae460 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f030d03f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03001abc10 con 0x561810c99390 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f02fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f02fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46846/0 (socket says 172.21.15.16:46846) 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f02fffff640 1 -- 172.21.15.16:0/4281629741 learned_addr learned my addr 172.21.15.16:0/4281629741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f0304867640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 0x561811064eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.191+0000 7f0305068640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811069fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 -- 172.21.15.16:0/4281629741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 msgr2=0x561811069fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811069fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 -- 172.21.15.16:0/4281629741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 msgr2=0x561811064eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 0x561811064eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 -- 172.21.15.16:0/4281629741 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56181106e090 con 0x561810c911b0 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f0304867640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 0x561811064eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02fffff640 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f030006ab20 tx=0x7f02f0009440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14876976ba1cb81c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02f67fc640 1 -- 172.21.15.16:0/4281629741 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f0031610 con 0x561810c911b0 2024-09-21T03:03:50.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02f67fc640 1 -- 172.21.15.16:0/4281629741 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f004b070 con 0x561810c911b0 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.192+0000 7f02f67fc640 1 -- 172.21.15.16:0/4281629741 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02f00046a0 con 0x561810c911b0 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- 172.21.15.16:0/4281629741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 msgr2=0x561811078fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f030006ab20 tx=0x7f02f0009440 comp rx=0 tx=0).stop 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- 172.21.15.16:0/4281629741 shutdown_connections 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811078fb0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810c99390 0x561811069fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 --2- 172.21.15.16:0/4281629741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810cae460 0x561811064eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- 172.21.15.16:0/4281629741 >> 172.21.15.16:0/4281629741 conn(0x56181106d490 msgr2=0x5618110647a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- 172.21.15.16:0/4281629741 shutdown_connections 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- 172.21.15.16:0/4281629741 wait complete. 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.193+0000 7f030d03f740 1 -- start start 2024-09-21T03:03:50.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 Processor -- start 2024-09-21T03:03:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c99390 0x561811070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810cae460 0x561811071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ac2e0 con 0x561810c911b0 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ad750 con 0x561810c99390 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f03001ac9b0 con 0x561810cae460 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f030d03f740 10 client.0 _ll_register_callbacks cb 0x7f02f8010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f0304867640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f0304867640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46860/0 (socket says 172.21.15.16:46860) 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f0304867640 1 -- 172.21.15.16:0/3605419785 learned_addr learned my addr 172.21.15.16:0/3605419785 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f0305068640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810cae460 0x561811071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.194+0000 7f02fffff640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c99390 0x561811070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 -- 172.21.15.16:0/3605419785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810cae460 msgr2=0x561811071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810cae460 0x561811071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 -- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c99390 msgr2=0x561811070dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c99390 0x561811070dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 -- 172.21.15.16:0/3605419785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561811071be0 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f0304867640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f02e80acfc0 tx=0x7f02e800af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb00c011108398a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f02dffff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02e80b5070 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f02e80b5210 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02f004b070 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f02e801c040 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f030d03f740 10 client.15798 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5618110e9ab0 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.195+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02f004b070 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f02e80f4040 con 0x561810c911b0 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 10 client.15798 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 10 client.15798 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 20 client.15798 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810cba070 0x5618110ebe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 4 client.15798 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561810fd58a0 con 0x561810cba070 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f02fffff640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810cba070 0x5618110ebe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f02dffff640 10 client.15798 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.196+0000 7f02fffff640 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810cba070 0x5618110ebe50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a59434b1b3d0c6d7 server_cookie=789e97ee8598ba69 in_seq=0 out_seq=0 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f02dffff640 1 -- 172.21.15.16:0/3605419785 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561810fd58a0 con 0x561810cba070 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f02dffff640 10 client.15798 handle_command_reply: tid=0 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 client.15798 shutdown 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 msgr2=0x561811075920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f02e80acfc0 tx=0x7f02e800af70 comp rx=0 tx=0).stop 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 shutdown_connections 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561810cba070 0x5618110ebe50 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561810cae460 0x561811071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561810c99390 0x561811070dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 --2- 172.21.15.16:0/3605419785 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561810c911b0 0x561811075920 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 >> 172.21.15.16:0/3605419785 conn(0x56181106d490 msgr2=0x56181106e260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f02dffff640 0 client.15798 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 shutdown_connections 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 1 -- 172.21.15.16:0/3605419785 wait complete. 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.197+0000 7f030d03f740 20 client.15798 trim_cache size 0 max 16384 2024-09-21T03:03:50.204 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 msgr2=0x7f02ec0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 0x7f02ec0b8c10 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009fd0 tx=0x7f02f80046c0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 msgr2=0x7f030019a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f02e800ad30 tx=0x7f02e8007590 comp rx=0 tx=0).stop 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 shutdown_connections 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f02ec0b6700 0x7f02ec0b8c10 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0300108d20 0x7f030019fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03001083d0 0x7f030019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 --2- 172.21.15.16:0/1244725087 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0300102ae0 0x7f030019a4c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 >> 172.21.15.16:0/1244725087 conn(0x7f0300079790 msgr2=0x7f0300101af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 shutdown_connections 2024-09-21T03:03:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:50.198+0000 7f0306acb640 1 -- 172.21.15.16:0/1244725087 wait complete. 2024-09-21T03:03:51.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:51 smithi018 ceph-mon[34587]: pgmap v425: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 413 MiB/s wr, 1.06k op/s 2024-09-21T03:03:51.452 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:51 smithi016 ceph-mon[31253]: pgmap v425: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 413 MiB/s wr, 1.06k op/s 2024-09-21T03:03:51.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:51 smithi123 ceph-mon[34645]: pgmap v425: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 413 MiB/s wr, 1.06k op/s 2024-09-21T03:03:52.516 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:03:52.517 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:03:53.043 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:53.340 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:53 smithi016 ceph-mon[31253]: pgmap v426: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 467 MiB/s wr, 893 op/s 2024-09-21T03:03:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:53 smithi018 ceph-mon[34587]: pgmap v426: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 467 MiB/s wr, 893 op/s 2024-09-21T03:03:53.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:53 smithi123 ceph-mon[34645]: pgmap v426: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 467 MiB/s wr, 893 op/s 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.046+0000 7f4793e74640 1 -- 172.21.15.16:0/2603026973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c11cc30 msgr2=0x7f478c11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.046+0000 7f4793e74640 1 --2- 172.21.15.16:0/2603026973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c11cc30 0x7f478c11f140 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f4784009be0 tx=0x7f478402a710 comp rx=0 tx=0).stop 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.046+0000 7f4793e74640 1 -- 172.21.15.16:0/2603026973 shutdown_connections 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.046+0000 7f4793e74640 1 --2- 172.21.15.16:0/2603026973 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c11cc30 0x7f478c11f140 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.046+0000 7f4793e74640 1 --2- 172.21.15.16:0/2603026973 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.047+0000 7f4793e74640 1 --2- 172.21.15.16:0/2603026973 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c111f40 0x7f478c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.047+0000 7f4793e74640 1 -- 172.21.15.16:0/2603026973 >> 172.21.15.16:0/2603026973 conn(0x7f478c070ec0 msgr2=0x7f478c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.055+0000 7f4793e74640 1 -- 172.21.15.16:0/2603026973 shutdown_connections 2024-09-21T03:03:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.055+0000 7f4793e74640 1 -- 172.21.15.16:0/2603026973 wait complete. 2024-09-21T03:03:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.057+0000 7f4793e74640 1 Processor -- start 2024-09-21T03:03:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.057+0000 7f4793e74640 1 -- start start 2024-09-21T03:03:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.057+0000 7f4793e74640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c11c470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f4793e74640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 0x7f478c116730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f4793e74640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c11cc30 0x7f478c116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f4793e74640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f478c112c40 con 0x7f478c111f40 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f4793e74640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f478c112ac0 con 0x7f478c11cc30 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f4793e74640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f478c112dc0 con 0x7f478c10a9e0 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.058+0000 7f479140f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 0x7f478c116730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.059+0000 7f479140f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 0x7f478c116730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46884/0 (socket says 172.21.15.16:46884) 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.059+0000 7f479140f640 1 -- 172.21.15.16:0/2849259405 learned_addr learned my addr 172.21.15.16:0/2849259405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:54.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.059+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c11c470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.061+0000 7f4792411640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c11cc30 0x7f478c116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.063+0000 7f4791c10640 1 -- 172.21.15.16:0/2849259405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c11cc30 msgr2=0x7f478c116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.063+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c11cc30 0x7f478c116cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.063+0000 7f4791c10640 1 -- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 msgr2=0x7f478c116730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.063+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 0x7f478c116730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.063+0000 7f4791c10640 1 -- 172.21.15.16:0/2849259405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f478c117510 con 0x7f478c10a9e0 2024-09-21T03:03:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.070+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c11c470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f477c00d6d0 tx=0x7f477c00db90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.071+0000 7f4782ffd640 1 -- 172.21.15.16:0/2849259405 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c017070 con 0x7f478c10a9e0 2024-09-21T03:03:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.071+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f478c07b6e0 con 0x7f478c10a9e0 2024-09-21T03:03:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.071+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f478c07bc20 con 0x7f478c10a9e0 2024-09-21T03:03:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.075+0000 7f4782ffd640 1 -- 172.21.15.16:0/2849259405 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f477c009e50 con 0x7f478c10a9e0 2024-09-21T03:03:54.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.075+0000 7f4782ffd640 1 -- 172.21.15.16:0/2849259405 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f477c00f3c0 con 0x7f478c10a9e0 2024-09-21T03:03:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.080+0000 7f479a3e8740 1 Processor -- start 2024-09-21T03:03:54.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.081+0000 7f479a3e8740 1 -- start start 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.081+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.081+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e18970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.081+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e18eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f479a3e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565a8d8ee60 con 0x5565a8a71350 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f479a3e8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565a8d7e290 con 0x5565a8a48490 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f479a3e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565a8d820a0 con 0x5565a8a50670 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f4792411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f4792411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46892/0 (socket says 172.21.15.16:46892) 2024-09-21T03:03:54.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.082+0000 7f4792411640 1 -- 172.21.15.16:0/2811539591 learned_addr learned my addr 172.21.15.16:0/2811539591 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f479140f640 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e18eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f4792411640 1 -- 172.21.15.16:0/2811539591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 msgr2=0x5565a8e18eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f4792411640 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e18eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f4792411640 1 -- 172.21.15.16:0/2811539591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 msgr2=0x5565a8e18970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f4792411640 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e18970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.083+0000 7f4792411640 1 -- 172.21.15.16:0/2811539591 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a8adc930 con 0x5565a8a71350 2024-09-21T03:03:54.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.084+0000 7f4792411640 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f478402ad00 tx=0x7f4784030500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73178ee6d2f14213 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.087+0000 7f4782ffd640 1 -- 172.21.15.16:0/2849259405 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f477c01a070 con 0x7f478c10a9e0 2024-09-21T03:03:54.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.088+0000 7f4782ffd640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 0x7f47780b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.089+0000 7f4782ffd640 1 -- 172.21.15.16:0/2849259405 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f477c0a8d50 con 0x7f478c10a9e0 2024-09-21T03:03:54.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.091+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 0x7f47780b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.093+0000 7f4791c10640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 0x7f47780b8d10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5565a8e170b0 tx=0x7f477c005cb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.104+0000 7f4777fff640 1 -- 172.21.15.16:0/2811539591 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784036040 con 0x5565a8a71350 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.105+0000 7f4777fff640 1 -- 172.21.15.16:0/2811539591 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4784003e20 con 0x5565a8a71350 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.105+0000 7f4777fff640 1 -- 172.21.15.16:0/2811539591 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47840319a0 con 0x5565a8a71350 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 -- 172.21.15.16:0/2811539591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 msgr2=0x5565a8e18430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f478402ad00 tx=0x7f4784030500 comp rx=0 tx=0).stop 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 -- 172.21.15.16:0/2811539591 shutdown_connections 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e18eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e18970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2811539591 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e18430 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.106+0000 7f479a3e8740 1 -- 172.21.15.16:0/2811539591 >> 172.21.15.16:0/2811539591 conn(0x5565a8a65740 msgr2=0x5565a8e16320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.107+0000 7f479a3e8740 1 -- 172.21.15.16:0/2811539591 shutdown_connections 2024-09-21T03:03:54.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.108+0000 7f479a3e8740 1 -- 172.21.15.16:0/2811539591 wait complete. 2024-09-21T03:03:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.109+0000 7f479a3e8740 1 -- start start 2024-09-21T03:03:54.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.109+0000 7f479a3e8740 1 Processor -- start 2024-09-21T03:03:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 0x5565a8e23fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a50670 0x5565a8e24520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e2ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5565a8bdf890 con 0x5565a8a71350 2024-09-21T03:03:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5565a8d18d40 con 0x5565a8a50670 2024-09-21T03:03:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.110+0000 7f479a3e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5565a8e26bb0 con 0x5565a8a48490 2024-09-21T03:03:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.111+0000 7f479a3e8740 10 client.0 _ll_register_callbacks cb 0x5565a8d7cc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:54.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.112+0000 7f4792411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e2ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.112+0000 7f4792411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e2ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46904/0 (socket says 172.21.15.16:46904) 2024-09-21T03:03:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.112+0000 7f4792411640 1 -- 172.21.15.16:0/2073596734 learned_addr learned my addr 172.21.15.16:0/2073596734 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:54.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.112+0000 7f4792411640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 0x5565a8e23fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.113+0000 7f4792411640 1 -- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a50670 msgr2=0x5565a8e24520 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:03:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.113+0000 7f4792411640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a50670 0x5565a8e24520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.113+0000 7f4792411640 1 -- 172.21.15.16:0/2073596734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 msgr2=0x5565a8e2ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.113+0000 7f4792411640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e2ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.113+0000 7f4792411640 1 -- 172.21.15.16:0/2073596734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a8e304a0 con 0x5565a8a48490 2024-09-21T03:03:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.116+0000 7f4792411640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 0x5565a8e23fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4784009b30 tx=0x7f478402a710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=993a42d59b0cfaee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.116+0000 7f47757fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:54.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.120+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784036070 con 0x5565a8a48490 2024-09-21T03:03:54.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.120+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4784003e20 con 0x5565a8a48490 2024-09-21T03:03:54.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.148+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4784003b80 con 0x5565a8a48490 2024-09-21T03:03:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.195+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4784031b00 con 0x5565a8a48490 2024-09-21T03:03:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.195+0000 7f479a3e8740 10 client.25450 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.195+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5565a8e32dd0 con 0x5565a8a48490 2024-09-21T03:03:54.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4784003e20 con 0x5565a8a48490 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4784053070 con 0x5565a8a48490 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 10 client.25450 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 10 client.25450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 20 client.25450 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a2fcd0 0x5565a8e36130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 4 client.25450 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5565a8d90db0 con 0x5565a8a2fcd0 2024-09-21T03:03:54.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.196+0000 7f479140f640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a2fcd0 0x5565a8e36130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.197+0000 7f47757fa640 10 client.25450 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:54.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.197+0000 7f479140f640 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a2fcd0 0x5565a8e36130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4de580b8f53736fa server_cookie=a4efbed85bf868cf in_seq=0 out_seq=0 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f47757fa640 1 -- 172.21.15.16:0/2073596734 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5565a8d90db0 con 0x5565a8a2fcd0 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f47757fa640 10 client.25450 handle_command_reply: tid=0 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 client.25450 shutdown 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 msgr2=0x5565a8e23fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 0x5565a8e23fe0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f4784009b30 tx=0x7f478402a710 comp rx=0 tx=0).stop 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 shutdown_connections 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a2fcd0 0x5565a8e36130 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a71350 0x5565a8e2ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f47757fa640 0 client.25450 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a50670 0x5565a8e24520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2073596734 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a48490 0x5565a8e23fe0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 >> 172.21.15.16:0/2073596734 conn(0x5565a8a65740 msgr2=0x5565a8e1b5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 shutdown_connections 2024-09-21T03:03:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 1 -- 172.21.15.16:0/2073596734 wait complete. 2024-09-21T03:03:54.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.198+0000 7f479a3e8740 20 client.25450 trim_cache size 0 max 16384 2024-09-21T03:03:54.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 Processor -- start 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 -- start start 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a50670 0x5565a8e22660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f477000c780 con 0x5565a8a50670 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f477000f000 con 0x5565a8a48490 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479a3e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f477000cdf0 con 0x5565a8a2fcd0 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f4792411640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a50670 0x5565a8e22660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479140f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f4792411640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479140f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57528/0 (socket says 172.21.15.16:57528) 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f479140f640 1 -- 172.21.15.16:0/2115228593 learned_addr learned my addr 172.21.15.16:0/2115228593 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:54.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.217+0000 7f4792411640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34488/0 (socket says 172.21.15.16:34488) 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 -- 172.21.15.16:0/2115228593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 msgr2=0x5565a8e1e550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 -- 172.21.15.16:0/2115228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a50670 msgr2=0x5565a8e22660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a50670 0x5565a8e22660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 -- 172.21.15.16:0/2115228593 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565a8e1a720 con 0x5565a8a2fcd0 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f479140f640 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f4792411640 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47840099a0 tx=0x7f4784000f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=14b1e31d82a415e8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f47777fe640 1 -- 172.21.15.16:0/2115228593 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4784036070 con 0x5565a8a2fcd0 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f47777fe640 1 -- 172.21.15.16:0/2115228593 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840308f0 con 0x5565a8a2fcd0 2024-09-21T03:03:54.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.218+0000 7f47777fe640 1 -- 172.21.15.16:0/2115228593 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f478403da70 con 0x5565a8a2fcd0 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 -- 172.21.15.16:0/2115228593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 msgr2=0x5565a8e2fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f47840099a0 tx=0x7f4784000f40 comp rx=0 tx=0).stop 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 -- 172.21.15.16:0/2115228593 shutdown_connections 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a50670 0x5565a8e22660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a48490 0x5565a8e1e550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 --2- 172.21.15.16:0/2115228593 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2fe30 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 -- 172.21.15.16:0/2115228593 >> 172.21.15.16:0/2115228593 conn(0x5565a8a65740 msgr2=0x5565a8e1ef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 -- 172.21.15.16:0/2115228593 shutdown_connections 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.219+0000 7f479a3e8740 1 -- 172.21.15.16:0/2115228593 wait complete. 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 -- start start 2024-09-21T03:03:54.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 Processor -- start 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2c890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e28490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f477000d4c0 con 0x5565a8a48490 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f477000e930 con 0x5565a8a2fcd0 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f477000db90 con 0x5565a8a50670 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479a3e8740 10 client.0 _ll_register_callbacks cb 0x7f4784030450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.220+0000 7f479140f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46936/0 (socket says 172.21.15.16:46936) 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 -- 172.21.15.16:0/516844669 learned_addr learned my addr 172.21.15.16:0/516844669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f4792411640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e28490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f4792411640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2c890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 -- 172.21.15.16:0/516844669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 msgr2=0x5565a8e28490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e28490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 -- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a2fcd0 msgr2=0x5565a8e2c890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2c890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 -- 172.21.15.16:0/516844669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565a8e9f960 con 0x5565a8a48490 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f4792411640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e28490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f479140f640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f478800be40 tx=0x7f4788013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1e36e2ead2c9a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.221+0000 7f47757fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4788027060 con 0x5565a8a48490 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f47840308f0 con 0x5565a8a48490 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4788011640 con 0x5565a8a48490 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4788011840 con 0x5565a8a48490 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f479a3e8740 10 client.15816 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:54.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5565a8ea0b50 con 0x5565a8a48490 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47840308f0 con 0x5565a8a48490 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.222+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4788010840 con 0x5565a8a48490 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 10 client.15816 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 10 client.15816 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 20 client.15816 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a71350 0x5565a8ea3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 4 client.15816 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5565a8d8c860 con 0x5565a8a71350 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f4792411640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a71350 0x5565a8ea3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f47757fa640 10 client.15816 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:54.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.223+0000 7f4792411640 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a71350 0x5565a8ea3320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ac9c8c8e014386a server_cookie=b3299d55958d002e in_seq=0 out_seq=0 2024-09-21T03:03:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.225+0000 7f47757fa640 1 -- 172.21.15.16:0/516844669 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5565a8d8c860 con 0x5565a8a71350 2024-09-21T03:03:54.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.225+0000 7f47757fa640 10 client.15816 handle_command_reply: tid=0 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.225+0000 7f479a3e8740 1 client.15816 shutdown 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.225+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 msgr2=0x5565a8e27d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.225+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f478800be40 tx=0x7f4788013830 comp rx=0 tx=0).stop 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 shutdown_connections 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5565a8a71350 0x5565a8ea3320 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5565a8a48490 0x5565a8e27d80 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5565a8a50670 0x5565a8e28490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 --2- 172.21.15.16:0/516844669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5565a8a2fcd0 0x5565a8e2c890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 >> 172.21.15.16:0/516844669 conn(0x5565a8a65740 msgr2=0x5565a8e15ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f47757fa640 0 client.15816 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 shutdown_connections 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 1 -- 172.21.15.16:0/516844669 wait complete. 2024-09-21T03:03:54.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.226+0000 7f479a3e8740 20 client.15816 trim_cache size 0 max 16384 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "fa8044a9-2474-4d92-84ff-5d4668e8b0cf", 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:54.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 msgr2=0x7f47780b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 0x7f47780b8d10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x5565a8e170b0 tx=0x7f477c005cb0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 msgr2=0x7f478c11c470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c11c470 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f477c00d6d0 tx=0x7f477c00db90 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 shutdown_connections 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f47780b6800 0x7f47780b8d10 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f478c111f40 0x7f478c116730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f478c11cc30 0x7f478c116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 --2- 172.21.15.16:0/2849259405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f478c10a9e0 0x7f478c11c470 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 >> 172.21.15.16:0/2849259405 conn(0x7f478c070ec0 msgr2=0x7f478c10ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 shutdown_connections 2024-09-21T03:03:54.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:54.227+0000 7f4793e74640 1 -- 172.21.15.16:0/2849259405 wait complete. 2024-09-21T03:03:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:54 smithi018 ceph-mon[34587]: pgmap v427: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 330 MiB/s wr, 198 op/s 2024-09-21T03:03:54.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:54 smithi123 ceph-mon[34645]: pgmap v427: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 330 MiB/s wr, 198 op/s 2024-09-21T03:03:54.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:54 smithi016 ceph-mon[31253]: pgmap v427: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 984 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 330 MiB/s wr, 198 op/s 2024-09-21T03:03:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:55 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:03:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:55 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:55 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:03:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:55 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.449 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:55 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:03:55.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:55 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:55 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:03:55.450 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:55 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.453 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:03:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:55 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:03:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:55 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:55 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:03:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:55 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:03:55.967 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:03:56.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:56 smithi123 ceph-mon[34645]: pgmap v428: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 466 MiB/s wr, 260 op/s 2024-09-21T03:03:56.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:56 smithi016 ceph-mon[31253]: pgmap v428: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 466 MiB/s wr, 260 op/s 2024-09-21T03:03:56.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:56 smithi018 ceph-mon[34587]: pgmap v428: 97 pgs: 97 active+clean; 29 GiB data, 89 GiB used, 983 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 466 MiB/s wr, 260 op/s 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.028+0000 7f59fe21f640 1 -- 172.21.15.16:0/2035083724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f810a9e0 msgr2=0x7f59f810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.028+0000 7f59fe21f640 1 --2- 172.21.15.16:0/2035083724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f810a9e0 0x7f59f810adf0 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f59e0009900 tx=0x7f59e002a410 comp rx=0 tx=0).stop 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.029+0000 7f59fe21f640 1 -- 172.21.15.16:0/2035083724 shutdown_connections 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.029+0000 7f59fe21f640 1 --2- 172.21.15.16:0/2035083724 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f810a9e0 0x7f59f810adf0 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.029+0000 7f59fe21f640 1 --2- 172.21.15.16:0/2035083724 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f811cc30 0x7f59f811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.029+0000 7f59fe21f640 1 --2- 172.21.15.16:0/2035083724 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.029+0000 7f59fe21f640 1 -- 172.21.15.16:0/2035083724 >> 172.21.15.16:0/2035083724 conn(0x7f59f8070ec0 msgr2=0x7f59f80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.034+0000 7f59fe21f640 1 -- 172.21.15.16:0/2035083724 shutdown_connections 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.034+0000 7f59fe21f640 1 -- 172.21.15.16:0/2035083724 wait complete. 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.035+0000 7f59fe21f640 1 Processor -- start 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.035+0000 7f59fe21f640 1 -- start start 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 0x7f59f811c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f81167b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 0x7f59f8116e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8112c80 con 0x7f59f811cc30 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8112b00 con 0x7f59f810a9e0 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59fe21f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59f8112e00 con 0x7f59f8111f40 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.036+0000 7f59f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 0x7f59f8116e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.037+0000 7f59f7fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 0x7f59f8116e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44722/0 (socket says 172.21.15.16:44722) 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.037+0000 7f59f7fff640 1 -- 172.21.15.16:0/882261142 learned_addr learned my addr 172.21.15.16:0/882261142 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.037+0000 7f59f77fe640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 0x7f59f811c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.037+0000 7f59eeffd640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f81167b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.042+0000 7f59f77fe640 1 -- 172.21.15.16:0/882261142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 msgr2=0x7f59f81167b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.042+0000 7f59f77fe640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f81167b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.042+0000 7f59f77fe640 1 -- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 msgr2=0x7f59f8116e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.042+0000 7f59f77fe640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 0x7f59f8116e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.042+0000 7f59f77fe640 1 -- 172.21.15.16:0/882261142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59f81bddb0 con 0x7f59f810a9e0 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.046+0000 7f59eeffd640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f81167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.046+0000 7f59f77fe640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 0x7f59f811c4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59e800be20 tx=0x7f59e8009f70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.047+0000 7f59f57fa640 1 -- 172.21.15.16:0/882261142 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e8019070 con 0x7f59f810a9e0 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.047+0000 7f59f57fa640 1 -- 172.21.15.16:0/882261142 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e80148f0 con 0x7f59f810a9e0 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.047+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59f81bdf80 con 0x7f59f810a9e0 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.047+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59f81be400 con 0x7f59f810a9e0 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.047+0000 7f59f57fa640 1 -- 172.21.15.16:0/882261142 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80128e0 con 0x7f59f810a9e0 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 Processor -- start 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 -- start start 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6875070 0x55f1d6c1c270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1d6b92e50 con 0x55f1d6875070 2024-09-21T03:03:57.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f1d6b82280 con 0x55f1d684c1b0 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.051+0000 7f5a04793740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1d6b86090 con 0x55f1d6854390 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.052+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.052+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41780/0 (socket says 172.21.15.16:41780) 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.052+0000 7f59eeffd640 1 -- 172.21.15.16:0/2343540745 learned_addr learned my addr 172.21.15.16:0/2343540745 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59f7fff640 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6875070 0x55f1d6c1c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59eeffd640 1 -- 172.21.15.16:0/2343540745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 msgr2=0x55f1d6c1c7b0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59eeffd640 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1c7b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59eeffd640 1 -- 172.21.15.16:0/2343540745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6875070 msgr2=0x55f1d6c1c270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59eeffd640 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6875070 0x55f1d6c1c270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.053+0000 7f59eeffd640 1 -- 172.21.15.16:0/2343540745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d68e0650 con 0x55f1d6854390 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.055+0000 7f59f57fa640 1 -- 172.21.15.16:0/882261142 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f59e801c070 con 0x7f59f810a9e0 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.055+0000 7f59eeffd640 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59e0031440 tx=0x7f59e0003680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=489d1cc06322223a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.055+0000 7f59edffb640 1 -- 172.21.15.16:0/2343540745 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0040070 con 0x55f1d6854390 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.056+0000 7f59edffb640 1 -- 172.21.15.16:0/2343540745 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e00038b0 con 0x55f1d6854390 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.056+0000 7f59edffb640 1 -- 172.21.15.16:0/2343540745 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0035040 con 0x55f1d6854390 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.057+0000 7f59f57fa640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 0x7f59e40b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.057+0000 7f59f57fa640 1 -- 172.21.15.16:0/882261142 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f59e80e6b30 con 0x7f59f810a9e0 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.057+0000 7f59f7fff640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 0x7f59e40b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 -- 172.21.15.16:0/2343540745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 msgr2=0x55f1d6c1ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f59e0031440 tx=0x7f59e0003680 comp rx=0 tx=0).stop 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 -- 172.21.15.16:0/2343540745 shutdown_connections 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c1ccf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1c7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 --2- 172.21.15.16:0/2343540745 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6875070 0x55f1d6c1c270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 -- 172.21.15.16:0/2343540745 >> 172.21.15.16:0/2343540745 conn(0x55f1d6869460 msgr2=0x55f1d6c1a160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.058+0000 7f5a04793740 1 -- 172.21.15.16:0/2343540745 shutdown_connections 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.059+0000 7f5a04793740 1 -- 172.21.15.16:0/2343540745 wait complete. 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.061+0000 7f59f7fff640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 0x7f59e40b8d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59f000ac00 tx=0x7f59f000a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.061+0000 7f5a04793740 1 -- start start 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.062+0000 7f5a04793740 1 Processor -- start 2024-09-21T03:03:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.063+0000 7f5a04793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.064+0000 7f5a04793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6854390 0x55f1d6c27eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.065+0000 7f5a04793740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 0x55f1d6c33d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.065+0000 7f5a04793740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f1d69e3700 con 0x55f1d6854390 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.066+0000 7f5a04793740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f1d6b1d110 con 0x55f1d684c1b0 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.066+0000 7f5a04793740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f1d6c2a9f0 con 0x55f1d6875070 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.066+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.066+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48998/0 (socket says 172.21.15.16:48998) 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.066+0000 7f59f77fe640 1 -- 172.21.15.16:0/840213000 learned_addr learned my addr 172.21.15.16:0/840213000 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.067+0000 7f5a04793740 10 client.0 _ll_register_callbacks cb 0x55f1d6b36990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.068+0000 7f59f7fff640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 0x55f1d6c33d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.071+0000 7f59f7fff640 1 -- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 msgr2=0x55f1d6c27970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.071+0000 7f59f7fff640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.071+0000 7f59f7fff640 1 -- 172.21.15.16:0/840213000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6854390 msgr2=0x55f1d6c27eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.071+0000 7f59f7fff640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6854390 0x55f1d6c27eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.071+0000 7f59f7fff640 1 -- 172.21.15.16:0/840213000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d6b4ac40 con 0x55f1d6875070 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.074+0000 7f59f77fe640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.076+0000 7f59f7fff640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 0x55f1d6c33d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59f000a9e0 tx=0x7f59f00046c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f67a78706b0c30fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.076+0000 7f59d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.077+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59f0005020 con 0x55f1d6875070 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.077+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e00038b0 con 0x55f1d6875070 2024-09-21T03:03:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.077+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f59f00051c0 con 0x55f1d6875070 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.078+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f59f00053c0 con 0x55f1d6875070 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.078+0000 7f5a04793740 10 client.25465 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.078+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f1d6c36820 con 0x55f1d6875070 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.078+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e00038b0 con 0x55f1d6875070 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.079+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f59f002c020 con 0x55f1d6875070 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.080+0000 7f5a04793740 10 client.25465 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.080+0000 7f5a04793740 10 client.25465 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.080+0000 7f5a04793740 20 client.25465 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.081+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d68339f0 0x55f1d6c3a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.081+0000 7f5a04793740 4 client.25465 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.081+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f1d6b94da0 con 0x55f1d68339f0 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.081+0000 7f59f77fe640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d68339f0 0x55f1d6c3a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.082+0000 7f59f77fe640 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d68339f0 0x55f1d6c3a240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=569aa94fad5a408f server_cookie=6659504d49ea4e7c in_seq=0 out_seq=0 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.082+0000 7f59d6ffd640 10 client.25465 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.082+0000 7f59d6ffd640 1 -- 172.21.15.16:0/840213000 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f1d6b94da0 con 0x55f1d68339f0 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.082+0000 7f59d6ffd640 10 client.25465 handle_command_reply: tid=0 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 client.25465 shutdown 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 msgr2=0x55f1d6c33d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 0x55f1d6c33d70 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f59f000a9e0 tx=0x7f59f00046c0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 shutdown_connections 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d68339f0 0x55f1d6c3a240 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.083+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6854390 0x55f1d6c27eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6875070 0x55f1d6c33d70 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f5a04793740 1 --2- 172.21.15.16:0/840213000 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c27970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 >> 172.21.15.16:0/840213000 conn(0x55f1d6869460 msgr2=0x55f1d6c1f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f59d6ffd640 0 client.25465 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:57.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 shutdown_connections 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.084+0000 7f5a04793740 1 -- 172.21.15.16:0/840213000 wait complete. 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.085+0000 7f5a04793740 20 client.25465 trim_cache size 0 max 16384 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.126+0000 7f5a04793740 1 Processor -- start 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f5a04793740 1 -- start start 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f5a04793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c27e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f5a04793740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1f370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c27e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c27e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44754/0 (socket says 172.21.15.16:44754) 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f59f77fe640 1 -- 172.21.15.16:0/522645169 learned_addr learned my addr 172.21.15.16:0/522645169 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:57.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.127+0000 7f5a04793740 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d6854390 0x55f1d6c1f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800c780 con 0x55f1d6869460 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800f000 con 0x55f1d6854390 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800cdf0 con 0x55f1d684c1b0 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59f7fff640 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d6854390 0x55f1d6c1f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 -- 172.21.15.16:0/522645169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d6854390 msgr2=0x55f1d6c1f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d6854390 0x55f1d6c1f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 -- 172.21.15.16:0/522645169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 msgr2=0x55f1d6c27e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c27e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 -- 172.21.15.16:0/522645169 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f1d6c268d0 con 0x55f1d684c1b0 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.128+0000 7f59eeffd640 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1f370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59e0031440 tx=0x7f59e002ab10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7b15ab87ab0ec60 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.129+0000 7f59ed7fa640 1 -- 172.21.15.16:0/522645169 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0035040 con 0x55f1d684c1b0 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.129+0000 7f59ed7fa640 1 -- 172.21.15.16:0/522645169 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e00314a0 con 0x55f1d684c1b0 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.129+0000 7f59ed7fa640 1 -- 172.21.15.16:0/522645169 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0040040 con 0x55f1d684c1b0 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 msgr2=0x55f1d6c1f370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1f370 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f59e0031440 tx=0x7f59e002ab10 comp rx=0 tx=0).stop 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 shutdown_connections 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d6854390 0x55f1d6c1f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d684c1b0 0x55f1d6c1f370 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 --2- 172.21.15.16:0/522645169 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c27e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 >> 172.21.15.16:0/522645169 conn(0x55f1d6c341f0 msgr2=0x55f1d6c27720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 shutdown_connections 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.130+0000 7f5a04793740 1 -- 172.21.15.16:0/522645169 wait complete. 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.131+0000 7f5a04793740 1 -- start start 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 Processor -- start 2024-09-21T03:03:57.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c2b9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c2bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f59f77fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:49010/0 (socket says 172.21.15.16:49010) 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c2b9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f59eeffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c2b9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:41816/0 (socket says 172.21.15.16:41816) 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f59f77fe640 1 -- 172.21.15.16:0/1155122834 learned_addr learned my addr 172.21.15.16:0/1155122834 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800d4c0 con 0x55f1d6869460 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800e930 con 0x55f1d684c1b0 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.132+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f59d800db90 con 0x55f1d6854390 2024-09-21T03:03:57.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f7fff640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c2bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f5a04793740 10 client.0 _ll_register_callbacks cb 0x7f59f000a850 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 -- 172.21.15.16:0/1155122834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 msgr2=0x55f1d6c2b9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c2b9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 -- 172.21.15.16:0/1155122834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 msgr2=0x55f1d6c2bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c2bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f1d6c2c740 con 0x55f1d684c1b0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59f77fe640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59e800cfd0 tx=0x7f59e8012a80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8f3ad1cc805ea913 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59d6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e80c7060 con 0x55f1d684c1b0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f59e80e44c0 con 0x55f1d684c1b0 2024-09-21T03:03:57.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.133+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59e00314a0 con 0x55f1d684c1b0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f59e8012340 con 0x55f1d684c1b0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 10 client.25511 fetch_fsmap learned FSMap version 20 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f1d6ca4c60 con 0x55f1d684c1b0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e00314a0 con 0x55f1d684c1b0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f59e8012b60 con 0x55f1d684c1b0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 10 client.25511 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 10 client.25511 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 20 client.25511 populate_metadata read hostname 'smithi016' 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d6875070 0x55f1d6ca7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 4 client.25511 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f1d6b90850 con 0x55f1d6875070 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.134+0000 7f59eeffd640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d6875070 0x55f1d6ca7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:03:57.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f59eeffd640 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d6875070 0x55f1d6ca7380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ba77eb75c42a3e92 server_cookie=d945fd859efa4a18 in_seq=0 out_seq=0 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f59d6ffd640 10 client.25511 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f59d6ffd640 1 -- 172.21.15.16:0/1155122834 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2274 (crc 0 0 0) 0x55f1d6b90850 con 0x55f1d6875070 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f59d6ffd640 10 client.25511 handle_command_reply: tid=0 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f5a04793740 1 client.25511 shutdown 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 msgr2=0x55f1d6c30540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.135+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f59e800cfd0 tx=0x7f59e8012a80 comp rx=0 tx=0).stop 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 shutdown_connections 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f1d6875070 0x55f1d6ca7380 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f1d6869460 0x55f1d6c2bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f59d6ffd640 0 client.25511 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f1d6854390 0x55f1d6c2b9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 --2- 172.21.15.16:0/1155122834 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f1d684c1b0 0x55f1d6c30540 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 >> 172.21.15.16:0/1155122834 conn(0x55f1d6c341f0 msgr2=0x55f1d6c1f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 shutdown_connections 2024-09-21T03:03:57.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 1 -- 172.21.15.16:0/1155122834 wait complete. 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "fa8044a9-2474-4d92-84ff-5d4668e8b0cf", 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:03:57.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:03:57.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.178 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:03:57.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.136+0000 7f5a04793740 20 client.25511 trim_cache size 0 max 16384 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 msgr2=0x7f59e40b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 0x7f59e40b8d90 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f59f000ac00 tx=0x7f59f000a2d0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 msgr2=0x7f59f811c4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 0x7f59f811c4f0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f59e800be20 tx=0x7f59e8009f70 comp rx=0 tx=0).stop 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 shutdown_connections 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f59e40b6880 0x7f59e40b8d90 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f59f811cc30 0x7f59f8116e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f59f8111f40 0x7f59f81167b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 --2- 172.21.15.16:0/882261142 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f59f810a9e0 0x7f59f811c4f0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:03:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 >> 172.21.15.16:0/882261142 conn(0x7f59f8070ec0 msgr2=0x7f59f810ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:03:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 shutdown_connections 2024-09-21T03:03:57.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:03:57.137+0000 7f59fe21f640 1 -- 172.21.15.16:0/882261142 wait complete. 2024-09-21T03:03:58.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:58 smithi016 ceph-mon[31253]: pgmap v429: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 396 MiB/s wr, 301 op/s 2024-09-21T03:03:58.385 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:03:58 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:03:58.521 INFO:tasks.cephfs.filesystem:scrub status for tag:fa8044a9-2474-4d92-84ff-5d4668e8b0cf - {'path': '/', 'tag': 'fa8044a9-2474-4d92-84ff-5d4668e8b0cf', 'options': 'recursive,force'} 2024-09-21T03:03:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:58 smithi018 ceph-mon[34587]: pgmap v429: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 396 MiB/s wr, 301 op/s 2024-09-21T03:03:58.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:03:58 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:03:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:58 smithi123 ceph-mon[34645]: pgmap v429: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 396 MiB/s wr, 301 op/s 2024-09-21T03:03:58.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:03:58 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:04:00.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:00 smithi018 ceph-mon[34587]: pgmap v430: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 395 MiB/s wr, 423 op/s 2024-09-21T03:04:00.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:00 smithi123 ceph-mon[34645]: pgmap v430: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 395 MiB/s wr, 423 op/s 2024-09-21T03:04:00.771 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:00 smithi016 ceph-mon[31253]: pgmap v430: 97 pgs: 97 active+clean; 29 GiB data, 90 GiB used, 983 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 395 MiB/s wr, 423 op/s 2024-09-21T03:04:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:02 smithi123 ceph-mon[34645]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 483 MiB/s wr, 919 op/s 2024-09-21T03:04:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:02 smithi016 ceph-mon[31253]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 483 MiB/s wr, 919 op/s 2024-09-21T03:04:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:02 smithi018 ceph-mon[34587]: pgmap v431: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 483 MiB/s wr, 919 op/s 2024-09-21T03:04:04.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:04 smithi016 ceph-mon[31253]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 409 MiB/s wr, 914 op/s 2024-09-21T03:04:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:04 smithi018 ceph-mon[34587]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 409 MiB/s wr, 914 op/s 2024-09-21T03:04:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:04 smithi123 ceph-mon[34645]: pgmap v432: 97 pgs: 97 active+clean; 30 GiB data, 91 GiB used, 982 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 409 MiB/s wr, 914 op/s 2024-09-21T03:04:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:06 smithi018 ceph-mon[34587]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 518 MiB/s wr, 1.00k op/s 2024-09-21T03:04:06.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:06 smithi123 ceph-mon[34645]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 518 MiB/s wr, 1.00k op/s 2024-09-21T03:04:07.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:06 smithi016 ceph-mon[31253]: pgmap v433: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 518 MiB/s wr, 1.00k op/s 2024-09-21T03:04:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:08 smithi018 ceph-mon[34587]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 409 MiB/s wr, 940 op/s 2024-09-21T03:04:08.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:08 smithi123 ceph-mon[34645]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 409 MiB/s wr, 940 op/s 2024-09-21T03:04:09.019 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:08 smithi016 ceph-mon[31253]: pgmap v434: 97 pgs: 97 active+clean; 30 GiB data, 92 GiB used, 981 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 409 MiB/s wr, 940 op/s 2024-09-21T03:04:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:10 smithi018 ceph-mon[34587]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 389 MiB/s wr, 960 op/s 2024-09-21T03:04:10.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:10 smithi123 ceph-mon[34645]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 389 MiB/s wr, 960 op/s 2024-09-21T03:04:11.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:10 smithi016 ceph-mon[31253]: pgmap v435: 97 pgs: 97 active+clean; 30 GiB data, 93 GiB used, 980 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 389 MiB/s wr, 960 op/s 2024-09-21T03:04:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:12 smithi018 ceph-mon[34587]: pgmap v436: 97 pgs: 97 active+clean; 32 GiB data, 96 GiB used, 976 GiB / 1.0 TiB avail; 778 KiB/s rd, 562 MiB/s wr, 1.54k op/s 2024-09-21T03:04:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:12 smithi123 ceph-mon[34645]: pgmap v436: 97 pgs: 97 active+clean; 32 GiB data, 96 GiB used, 976 GiB / 1.0 TiB avail; 778 KiB/s rd, 562 MiB/s wr, 1.54k op/s 2024-09-21T03:04:13.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:12 smithi016 ceph-mon[31253]: pgmap v436: 97 pgs: 97 active+clean; 32 GiB data, 96 GiB used, 976 GiB / 1.0 TiB avail; 778 KiB/s rd, 562 MiB/s wr, 1.54k op/s 2024-09-21T03:04:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:14 smithi018 ceph-mon[34587]: pgmap v437: 97 pgs: 97 active+clean; 32 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 731 KiB/s rd, 473 MiB/s wr, 1.17k op/s 2024-09-21T03:04:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:14 smithi123 ceph-mon[34645]: pgmap v437: 97 pgs: 97 active+clean; 32 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 731 KiB/s rd, 473 MiB/s wr, 1.17k op/s 2024-09-21T03:04:15.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:14 smithi016 ceph-mon[31253]: pgmap v437: 97 pgs: 97 active+clean; 32 GiB data, 97 GiB used, 976 GiB / 1.0 TiB avail; 731 KiB/s rd, 473 MiB/s wr, 1.17k op/s 2024-09-21T03:04:16.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:16 smithi018 ceph-mon[34587]: pgmap v438: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 561 MiB/s wr, 1.97k op/s 2024-09-21T03:04:16.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:16 smithi123 ceph-mon[34645]: pgmap v438: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 561 MiB/s wr, 1.97k op/s 2024-09-21T03:04:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:16 smithi016 ceph-mon[31253]: pgmap v438: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 561 MiB/s wr, 1.97k op/s 2024-09-21T03:04:18.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:18 smithi018 ceph-mon[34587]: pgmap v439: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 690 KiB/s rd, 467 MiB/s wr, 2.14k op/s 2024-09-21T03:04:18.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:18 smithi123 ceph-mon[34645]: pgmap v439: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 690 KiB/s rd, 467 MiB/s wr, 2.14k op/s 2024-09-21T03:04:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:18 smithi016 ceph-mon[31253]: pgmap v439: 97 pgs: 97 active+clean; 32 GiB data, 98 GiB used, 975 GiB / 1.0 TiB avail; 690 KiB/s rd, 467 MiB/s wr, 2.14k op/s 2024-09-21T03:04:20.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:20 smithi018 ceph-mon[34587]: pgmap v440: 97 pgs: 97 active+clean; 32 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 431 KiB/s rd, 429 MiB/s wr, 2.13k op/s 2024-09-21T03:04:20.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:20 smithi123 ceph-mon[34645]: pgmap v440: 97 pgs: 97 active+clean; 32 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 431 KiB/s rd, 429 MiB/s wr, 2.13k op/s 2024-09-21T03:04:21.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:20 smithi016 ceph-mon[31253]: pgmap v440: 97 pgs: 97 active+clean; 32 GiB data, 99 GiB used, 974 GiB / 1.0 TiB avail; 431 KiB/s rd, 429 MiB/s wr, 2.13k op/s 2024-09-21T03:04:22.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:22 smithi018 ceph-mon[34587]: pgmap v441: 97 pgs: 97 active+clean; 33 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 500 MiB/s wr, 2.25k op/s 2024-09-21T03:04:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:22 smithi123 ceph-mon[34645]: pgmap v441: 97 pgs: 97 active+clean; 33 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 500 MiB/s wr, 2.25k op/s 2024-09-21T03:04:23.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:22 smithi016 ceph-mon[31253]: pgmap v441: 97 pgs: 97 active+clean; 33 GiB data, 100 GiB used, 972 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 500 MiB/s wr, 2.25k op/s 2024-09-21T03:04:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:24 smithi123 ceph-mon[34645]: pgmap v442: 97 pgs: 97 active+clean; 33 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 328 MiB/s wr, 1.62k op/s 2024-09-21T03:04:25.033 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:24 smithi016 ceph-mon[31253]: pgmap v442: 97 pgs: 97 active+clean; 33 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 328 MiB/s wr, 1.62k op/s 2024-09-21T03:04:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:24 smithi018 ceph-mon[34587]: pgmap v442: 97 pgs: 97 active+clean; 33 GiB data, 101 GiB used, 972 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 328 MiB/s wr, 1.62k op/s 2024-09-21T03:04:26.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:26 smithi123 ceph-mon[34645]: pgmap v443: 97 pgs: 97 active+clean; 35 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 493 MiB/s wr, 2.24k op/s 2024-09-21T03:04:27.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:26 smithi016 ceph-mon[31253]: pgmap v443: 97 pgs: 97 active+clean; 35 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 493 MiB/s wr, 2.24k op/s 2024-09-21T03:04:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:26 smithi018 ceph-mon[34587]: pgmap v443: 97 pgs: 97 active+clean; 35 GiB data, 107 GiB used, 966 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 493 MiB/s wr, 2.24k op/s 2024-09-21T03:04:28.522 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:04:28.818 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:28 smithi016 ceph-mon[31253]: pgmap v444: 97 pgs: 97 active+clean; 36 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 455 MiB/s wr, 1.60k op/s 2024-09-21T03:04:29.023 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:28 smithi018 ceph-mon[34587]: pgmap v444: 97 pgs: 97 active+clean; 36 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 455 MiB/s wr, 1.60k op/s 2024-09-21T03:04:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:28 smithi123 ceph-mon[34645]: pgmap v444: 97 pgs: 97 active+clean; 36 GiB data, 108 GiB used, 965 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 455 MiB/s wr, 1.60k op/s 2024-09-21T03:04:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 -- 172.21.15.16:0/1548821553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 msgr2=0x7f1b50107f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 --2- 172.21.15.16:0/1548821553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b50107f40 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f1b44009900 tx=0x7f1b4402a410 comp rx=0 tx=0).stop 2024-09-21T03:04:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 -- 172.21.15.16:0/1548821553 shutdown_connections 2024-09-21T03:04:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 --2- 172.21.15.16:0/1548821553 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b50107f40 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 --2- 172.21.15.16:0/1548821553 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b5010ecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 --2- 172.21.15.16:0/1548821553 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 0x7f1b50102600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 -- 172.21.15.16:0/1548821553 >> 172.21.15.16:0/1548821553 conn(0x7f1b50077c60 msgr2=0x7f1b500780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 -- 172.21.15.16:0/1548821553 shutdown_connections 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.958+0000 7f1b56375640 1 -- 172.21.15.16:0/1548821553 wait complete. 2024-09-21T03:04:29.961 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.960+0000 7f1b56375640 1 Processor -- start 2024-09-21T03:04:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.960+0000 7f1b56375640 1 -- start start 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.960+0000 7f1b56375640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 0x7f1b501980b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b56375640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b56375640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b50198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b56375640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50111910 con 0x7f1b50107b30 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b56375640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50111790 con 0x7f1b50108480 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b56375640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50111a90 con 0x7f1b501021f0 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39268/0 (socket says 172.21.15.16:39268) 2024-09-21T03:04:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/570195979 learned_addr learned my addr 172.21.15.16:0/570195979 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b54912640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b50198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.961+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 0x7f1b501980b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/570195979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 msgr2=0x7f1b501980b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 0x7f1b501980b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/570195979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 msgr2=0x7f1b50198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b50198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/570195979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b501a8750 con 0x7f1b50107b30 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b54912640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b50198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.962+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b44031440 tx=0x7f1b44003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.963+0000 7f1b4d7fa640 1 -- 172.21.15.16:0/570195979 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4403f070 con 0x7f1b50107b30 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.963+0000 7f1b4d7fa640 1 -- 172.21.15.16:0/570195979 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b44031b50 con 0x7f1b50107b30 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.963+0000 7f1b4d7fa640 1 -- 172.21.15.16:0/570195979 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b44035030 con 0x7f1b50107b30 2024-09-21T03:04:29.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.963+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b501a89e0 con 0x7f1b50107b30 2024-09-21T03:04:29.966 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.963+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b501a8e70 con 0x7f1b50107b30 2024-09-21T03:04:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.969+0000 7f1b5c8e9740 1 Processor -- start 2024-09-21T03:04:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.969+0000 7f1b5c8e9740 1 -- start start 2024-09-21T03:04:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01de1650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a111b0 0x55ef01de1b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b4d7fa640 1 -- 172.21.15.16:0/570195979 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1b44042070 con 0x7f1b50107b30 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef01d57e50 con 0x55ef01a3a070 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ef01d47280 con 0x55ef01a111b0 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef01d4b090 con 0x55ef01a19390 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.970+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:33118/0 (socket says 172.21.15.16:33118) 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/4287701312 learned_addr learned my addr 172.21.15.16:0/4287701312 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/4287701312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a111b0 msgr2=0x55ef01de1b90 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a111b0 0x55ef01de1b90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/4287701312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 msgr2=0x55ef01de1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01de1650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/4287701312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef01aa5650 con 0x55ef01a19390 2024-09-21T03:04:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.971+0000 7f1b4d7fa640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 0x7f1b380b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44104da0 tx=0x7f1b4410dcb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1b8c4246bca238a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:29.974 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b54912640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 0x7f1b380b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b4d7fa640 1 -- 172.21.15.16:0/570195979 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f1b44107750 con 0x7f1b50107b30 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b367fc640 1 -- 172.21.15.16:0/4287701312 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440c9c60 con 0x55ef01a19390 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b367fc640 1 -- 172.21.15.16:0/4287701312 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b440cd120 con 0x55ef01a19390 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.972+0000 7f1b367fc640 1 -- 172.21.15.16:0/4287701312 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b440f0630 con 0x55ef01a19390 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/4287701312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 msgr2=0x55ef01de20d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b54912640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 0x7f1b380b8cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b48009840 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f1b44104da0 tx=0x7f1b4410dcb0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/4287701312 shutdown_connections 2024-09-21T03:04:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01de20d0 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a111b0 0x55ef01de1b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/4287701312 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01de1650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.974+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/4287701312 >> 172.21.15.16:0/4287701312 conn(0x55ef01a2e460 msgr2=0x55ef01ddf540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.975+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/4287701312 shutdown_connections 2024-09-21T03:04:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.975+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/4287701312 wait complete. 2024-09-21T03:04:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.975+0000 7f1b5c8e9740 1 -- start start 2024-09-21T03:04:29.978 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.976+0000 7f1b5c8e9740 1 Processor -- start 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.977+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01dfdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.977+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 0x55ef01dfe1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.977+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01dfe750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ef01ba8700 con 0x55ef01a3a070 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ef01ce2110 con 0x55ef01a19390 2024-09-21T03:04:29.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01dfdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ef01e00b50 con 0x55ef01a111b0 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 0x55ef01dfe1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 0x55ef01dfe1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44950/0 (socket says 172.21.15.16:44950) 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4f7fe640 1 -- 172.21.15.16:0/683968513 learned_addr learned my addr 172.21.15.16:0/683968513 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b5c8e9740 10 client.0 _ll_register_callbacks cb 0x55ef01cfb990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b54912640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01dfe750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 -- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 msgr2=0x55ef01dfe1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 0x55ef01dfe1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 -- 172.21.15.16:0/683968513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 msgr2=0x55ef01dfe750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01dfe750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b4ffff640 1 -- 172.21.15.16:0/683968513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef01d0fc40 con 0x55ef01a111b0 2024-09-21T03:04:29.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.978+0000 7f1b54912640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01dfe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.979+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01dfdcb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b50199b00 tx=0x7f1b4000eb60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2dd1ef25ac1003a3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.979+0000 7f1b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.979+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40004250 con 0x55ef01a111b0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.979+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b440cd120 con 0x55ef01a111b0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.979+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f1b400043f0 con 0x55ef01a111b0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.980+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1b4000d8e0 con 0x55ef01a111b0 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.980+0000 7f1b5c8e9740 10 client.25483 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:29.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.980+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ef01e0c840 con 0x55ef01a111b0 2024-09-21T03:04:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.980+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b440cd120 con 0x55ef01a111b0 2024-09-21T03:04:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.980+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1b4001f020 con 0x55ef01a111b0 2024-09-21T03:04:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.981+0000 7f1b5c8e9740 10 client.25483 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.981+0000 7f1b5c8e9740 10 client.25483 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:29.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.981+0000 7f1b5c8e9740 20 client.25483 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.981+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef019f89f0 0x55ef01e0fd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.982+0000 7f1b5c8e9740 4 client.25483 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.982+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef01d59da0 con 0x55ef019f89f0 2024-09-21T03:04:29.984 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.983+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef019f89f0 0x55ef01e0fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.983+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef019f89f0 0x55ef01e0fd90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d40abf98cb303ed server_cookie=b536946dcfbdf8de in_seq=0 out_seq=0 2024-09-21T03:04:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.983+0000 7f1b2f7fe640 10 client.25483 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:29.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.984+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/683968513 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ef01d59da0 con 0x55ef019f89f0 2024-09-21T03:04:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.984+0000 7f1b2f7fe640 10 client.25483 handle_command_reply: tid=0 2024-09-21T03:04:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.984+0000 7f1b5c8e9740 1 client.25483 shutdown 2024-09-21T03:04:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 msgr2=0x55ef01dfdcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01dfdcb0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f1b50199b00 tx=0x7f1b4000eb60 comp rx=0 tx=0).stop 2024-09-21T03:04:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 shutdown_connections 2024-09-21T03:04:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef019f89f0 0x55ef01e0fd90 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a3a070 0x55ef01dfe750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.985+0000 7f1b2f7fe640 0 client.25483 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef01a19390 0x55ef01dfe1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/683968513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01dfdcb0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 >> 172.21.15.16:0/683968513 conn(0x55ef01a2e460 msgr2=0x55ef01de3040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 shutdown_connections 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/683968513 wait complete. 2024-09-21T03:04:29.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:29.986+0000 7f1b5c8e9740 20 client.25483 trim_cache size 0 max 16384 2024-09-21T03:04:30.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.019+0000 7f1b5c8e9740 1 Processor -- start 2024-09-21T03:04:30.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.019+0000 7f1b5c8e9740 1 -- start start 2024-09-21T03:04:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.019+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01dfe170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01de6f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 0x55ef01dfaea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01dfe170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b54912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 0x55ef01dfaea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b54912640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 0x55ef01dfaea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39290/0 (socket says 172.21.15.16:39290) 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b54912640 1 -- 172.21.15.16:0/697855372 learned_addr learned my addr 172.21.15.16:0/697855372 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50073730 con 0x55ef01a19390 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501a9950 con 0x55ef019f89f0 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.020+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50073da0 con 0x55ef01a111b0 2024-09-21T03:04:30.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01de6f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4ffff640 1 -- 172.21.15.16:0/697855372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 msgr2=0x55ef01de6f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01de6f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4ffff640 1 -- 172.21.15.16:0/697855372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 msgr2=0x55ef01dfaea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 0x55ef01dfaea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.021+0000 7f1b4ffff640 1 -- 172.21.15.16:0/697855372 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef01dfcc10 con 0x55ef019f89f0 2024-09-21T03:04:30.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.022+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01dfe170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1b40003f70 tx=0x7f1b4000ae30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec98ff85aefef1ea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.022+0000 7f1b35ffb640 1 -- 172.21.15.16:0/697855372 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40000e60 con 0x55ef019f89f0 2024-09-21T03:04:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.022+0000 7f1b35ffb640 1 -- 172.21.15.16:0/697855372 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b40032030 con 0x55ef019f89f0 2024-09-21T03:04:30.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.022+0000 7f1b35ffb640 1 -- 172.21.15.16:0/697855372 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b40018b10 con 0x55ef019f89f0 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 msgr2=0x55ef01dfe170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01dfe170 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f1b40003f70 tx=0x7f1b4000ae30 comp rx=0 tx=0).stop 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 shutdown_connections 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a19390 0x55ef01dfaea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a111b0 0x55ef01de6f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/697855372 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01dfe170 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.024+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 >> 172.21.15.16:0/697855372 conn(0x55ef01a2e460 msgr2=0x55ef01dfda60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:30.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.025+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 shutdown_connections 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.025+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/697855372 wait complete. 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.025+0000 7f1b5c8e9740 1 -- start start 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.026+0000 7f1b5c8e9740 1 Processor -- start 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.027+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.027+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 0x55ef01e02be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.028+0000 7f1b5c8e9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01e79790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.028+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50074470 con 0x55ef01a111b0 2024-09-21T03:04:30.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.028+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1b501a9280 con 0x55ef019f89f0 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.028+0000 7f1b5c8e9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1b50074b40 con 0x55ef01a19390 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.028+0000 7f1b5c8e9740 10 client.0 _ll_register_callbacks cb 0x7f1b4000a810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.029+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.029+0000 7f1b4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44964/0 (socket says 172.21.15.16:44964) 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.029+0000 7f1b4ffff640 1 -- 172.21.15.16:0/3590512852 learned_addr learned my addr 172.21.15.16:0/3590512852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.029+0000 7f1b4ffff640 1 -- 172.21.15.16:0/3590512852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 msgr2=0x55ef01e79790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.029+0000 7f1b54912640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01e79790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.030+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 0x55ef01e02be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.030+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01e79790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.030+0000 7f1b4ffff640 1 -- 172.21.15.16:0/3590512852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 msgr2=0x55ef01e02be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:30.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.030+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 0x55ef01e02be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.030+0000 7f1b4ffff640 1 -- 172.21.15.16:0/3590512852 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef01e79e00 con 0x55ef019f89f0 2024-09-21T03:04:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.031+0000 7f1b4ffff640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000a490 tx=0x7f1b400187a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4634260e7a8dba4d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.031+0000 7f1b2f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:30.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.032+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b4003c060 con 0x55ef019f89f0 2024-09-21T03:04:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.032+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f1b400188d0 con 0x55ef019f89f0 2024-09-21T03:04:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.032+0000 7f1b54912640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01e79790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.032+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b40032030 con 0x55ef019f89f0 2024-09-21T03:04:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.034+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 0x55ef01e02be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:30.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.037+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1b4000ac60 con 0x55ef019f89f0 2024-09-21T03:04:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.042+0000 7f1b5c8e9740 10 client.25526 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:30.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.042+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ef01e7b070 con 0x55ef019f89f0 2024-09-21T03:04:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.047+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b400035f0 con 0x55ef019f89f0 2024-09-21T03:04:30.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.047+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1b4000b040 con 0x55ef019f89f0 2024-09-21T03:04:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 10 client.25526 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 10 client.25526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:30.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 20 client.25526 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef01a3a070 0x55ef01e7d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 4 client.25526 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:04:30.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.049+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f1b40032030 con 0x55ef01a3a070 2024-09-21T03:04:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.056+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef01a3a070 0x55ef01e7d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.071+0000 7f1b4f7fe640 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef01a3a070 0x55ef01e7d720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a14063558f7429cf server_cookie=2d6b64b41593e9eb in_seq=0 out_seq=0 2024-09-21T03:04:30.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.071+0000 7f1b2f7fe640 10 client.25526 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.077+0000 7f1b2f7fe640 1 -- 172.21.15.16:0/3590512852 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2078 (crc 0 0 0) 0x7f1b40032030 con 0x55ef01a3a070 2024-09-21T03:04:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.077+0000 7f1b2f7fe640 10 client.25526 handle_command_reply: tid=0 2024-09-21T03:04:30.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.077+0000 7f1b5c8e9740 1 client.25526 shutdown 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.081+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 msgr2=0x55ef01e02620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.081+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f1b4000a490 tx=0x7f1b400187a0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 shutdown_connections 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ef01a3a070 0x55ef01e7d720 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ef01a111b0 0x55ef01e02be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b2f7fe640 0 client.25526 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ef01a19390 0x55ef01e79790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 --2- 172.21.15.16:0/3590512852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ef019f89f0 0x55ef01e02620 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.082+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 >> 172.21.15.16:0/3590512852 conn(0x55ef01a2e460 msgr2=0x55ef01de6f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:30.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.089+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 shutdown_connections 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:04:30.109 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:30.110 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.099+0000 7f1b5c8e9740 1 -- 172.21.15.16:0/3590512852 wait complete. 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.099+0000 7f1b5c8e9740 20 client.25526 trim_cache size 0 max 16384 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.100+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 msgr2=0x7f1b380b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.100+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 0x7f1b380b8cc0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f1b48009fd0 tx=0x7f1b48009840 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.100+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 msgr2=0x7f1b501985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.100+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f1b44031440 tx=0x7f1b44003c00 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 shutdown_connections 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1b380b67b0 0x7f1b380b8cc0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1b50107b30 0x7f1b501985f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1b50108480 0x7f1b50198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 --2- 172.21.15.16:0/570195979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b501021f0 0x7f1b501980b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.103+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 >> 172.21.15.16:0/570195979 conn(0x7f1b50077c60 msgr2=0x7f1b50103b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:30.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.106+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 shutdown_connections 2024-09-21T03:04:30.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:30.106+0000 7f1b56375640 1 -- 172.21.15.16:0/570195979 wait complete. 2024-09-21T03:04:31.158 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:30 smithi016 ceph-mon[31253]: pgmap v445: 97 pgs: 97 active+clean; 36 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 469 MiB/s wr, 1.45k op/s 2024-09-21T03:04:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:30 smithi018 ceph-mon[34587]: pgmap v445: 97 pgs: 97 active+clean; 36 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 469 MiB/s wr, 1.45k op/s 2024-09-21T03:04:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:30 smithi123 ceph-mon[34645]: pgmap v445: 97 pgs: 97 active+clean; 36 GiB data, 109 GiB used, 964 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 469 MiB/s wr, 1.45k op/s 2024-09-21T03:04:31.373 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:04:31.374 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:04:31.972 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.617+0000 7f0aaea81640 1 -- 172.21.15.16:0/1251503914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8102f50 msgr2=0x7f0aa8107480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.617+0000 7f0aaea81640 1 --2- 172.21.15.16:0/1251503914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8102f50 0x7f0aa8107480 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4009930 tx=0x7f0aa402a440 comp rx=0 tx=0).stop 2024-09-21T03:04:32.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.618+0000 7f0aaea81640 1 -- 172.21.15.16:0/1251503914 shutdown_connections 2024-09-21T03:04:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.618+0000 7f0aaea81640 1 --2- 172.21.15.16:0/1251503914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8102f50 0x7f0aa8107480 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.618+0000 7f0aaea81640 1 --2- 172.21.15.16:0/1251503914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8107a40 0x7f0aa8107f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.618+0000 7f0aaea81640 1 --2- 172.21.15.16:0/1251503914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102600 0x7f0aa8102a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.618+0000 7f0aaea81640 1 -- 172.21.15.16:0/1251503914 >> 172.21.15.16:0/1251503914 conn(0x7f0aa80fe350 msgr2=0x7f0aa81007a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:32.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.619+0000 7f0aaea81640 1 -- 172.21.15.16:0/1251503914 shutdown_connections 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.619+0000 7f0aaea81640 1 -- 172.21.15.16:0/1251503914 wait complete. 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.621+0000 7f0aaea81640 1 Processor -- start 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.621+0000 7f0aaea81640 1 -- start start 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.622+0000 7f0aaea81640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 0x7f0aa8116cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.622+0000 7f0aaea81640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102f50 0x7f0aa8110fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:32.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.623+0000 7f0aaea81640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 0x7f0aa8111770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.623+0000 7f0aaea81640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa8109350 con 0x7f0aa8107a40 2024-09-21T03:04:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.623+0000 7f0aaea81640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81091d0 con 0x7f0aa8102600 2024-09-21T03:04:32.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.623+0000 7f0aaea81640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0aa81094d0 con 0x7f0aa8102f50 2024-09-21T03:04:32.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.625+0000 7f0aada7f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 0x7f0aa8116cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.626+0000 7f0aada7f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 0x7f0aa8116cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:44974/0 (socket says 172.21.15.16:44974) 2024-09-21T03:04:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.626+0000 7f0aada7f640 1 -- 172.21.15.16:0/888104080 learned_addr learned my addr 172.21.15.16:0/888104080 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.626+0000 7f0aae280640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 0x7f0aa8111770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:32.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.626+0000 7f0aad27e640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102f50 0x7f0aa8110fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.631+0000 7f0aae280640 1 -- 172.21.15.16:0/888104080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102f50 msgr2=0x7f0aa8110fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:32.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.631+0000 7f0aae280640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102f50 0x7f0aa8110fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.631+0000 7f0aae280640 1 -- 172.21.15.16:0/888104080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 msgr2=0x7f0aa8116cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.631+0000 7f0aae280640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 0x7f0aa8116cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.631+0000 7f0aae280640 1 -- 172.21.15.16:0/888104080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0aa81b5460 con 0x7f0aa8107a40 2024-09-21T03:04:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.636+0000 7f0aae280640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 0x7f0aa8111770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b3e0 tx=0x7f0a9800b8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.637+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98015020 con 0x7f0aa8107a40 2024-09-21T03:04:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.637+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a98011ce0 con 0x7f0aa8107a40 2024-09-21T03:04:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.637+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a98010e00 con 0x7f0aa8107a40 2024-09-21T03:04:32.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.637+0000 7f0aaea81640 1 -- 172.21.15.16:0/888104080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0aa81b56f0 con 0x7f0aa8107a40 2024-09-21T03:04:32.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.640+0000 7f0aaea81640 1 -- 172.21.15.16:0/888104080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0aa81b5bb0 con 0x7f0aa8107a40 2024-09-21T03:04:32.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.648+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0aa806bf20 con 0x7f0aa8107a40 2024-09-21T03:04:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.659+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a98018020 con 0x7f0aa8107a40 2024-09-21T03:04:32.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.661+0000 7f0a9effd640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 0x7f0a940b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.666+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f0a980e7360 con 0x7f0aa8107a40 2024-09-21T03:04:32.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.666+0000 7f0aada7f640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 0x7f0a940b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:32.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.668+0000 7f0aada7f640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 0x7f0a940b8d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8116a40 tx=0x7f0aa000b040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:32.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.706+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f0a980ad8f0 con 0x7f0aa8107a40 2024-09-21T03:04:32.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.991+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0aa81122b0 con 0x7f0aa8107a40 2024-09-21T03:04:32.993 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:32 smithi016 ceph-mon[31253]: pgmap v446: 97 pgs: 97 active+clean; 37 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 638 MiB/s wr, 1.79k op/s 2024-09-21T03:04:32.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.992+0000 7f0a9effd640 1 -- 172.21.15.16:0/888104080 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f0a980b1cd0 con 0x7f0aa8107a40 2024-09-21T03:04:32.994 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:04:32.994 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:04:32.995 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:04:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 msgr2=0x7f0a940b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 0x7f0a940b8d90 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8116a40 tx=0x7f0aa000b040 comp rx=0 tx=0).stop 2024-09-21T03:04:32.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 msgr2=0x7f0aa8111770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 0x7f0aa8111770 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f0a9800b3e0 tx=0x7f0a9800b8a0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 shutdown_connections 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a940b6880 0x7f0a940b8d90 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0aa8107a40 0x7f0aa8111770 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0aa8102f50 0x7f0aa8110fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 --2- 172.21.15.16:0/888104080 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0aa8102600 0x7f0aa8116cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 >> 172.21.15.16:0/888104080 conn(0x7f0aa80fe350 msgr2=0x7f0aa8105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 shutdown_connections 2024-09-21T03:04:32.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:32.995+0000 7f0a9cff9640 1 -- 172.21.15.16:0/888104080 wait complete. 2024-09-21T03:04:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:32 smithi018 ceph-mon[34587]: pgmap v446: 97 pgs: 97 active+clean; 37 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 638 MiB/s wr, 1.79k op/s 2024-09-21T03:04:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:32 smithi123 ceph-mon[34645]: pgmap v446: 97 pgs: 97 active+clean; 37 GiB data, 113 GiB used, 960 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 638 MiB/s wr, 1.79k op/s 2024-09-21T03:04:34.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/888104080' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:34.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/888104080' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:34.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/888104080' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:34.371 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:04:34.669 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:35.161 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:34 smithi016 ceph-mon[31253]: pgmap v447: 97 pgs: 97 active+clean; 38 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 556 MiB/s wr, 1.61k op/s 2024-09-21T03:04:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:34 smithi018 ceph-mon[34587]: pgmap v447: 97 pgs: 97 active+clean; 38 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 556 MiB/s wr, 1.61k op/s 2024-09-21T03:04:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:34 smithi123 ceph-mon[34645]: pgmap v447: 97 pgs: 97 active+clean; 38 GiB data, 114 GiB used, 959 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 556 MiB/s wr, 1.61k op/s 2024-09-21T03:04:35.564 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 -- 172.21.15.16:0/3083236616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e4103b80 msgr2=0x7f98e4103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 --2- 172.21.15.16:0/3083236616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e4103b80 0x7f98e4103f90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f98d4009900 tx=0x7f98d402a410 comp rx=0 tx=0).stop 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 -- 172.21.15.16:0/3083236616 shutdown_connections 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 --2- 172.21.15.16:0/3083236616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4109e10 0x7f98e4110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 --2- 172.21.15.16:0/3083236616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e41094c0 0x7f98e41098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 --2- 172.21.15.16:0/3083236616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e4103b80 0x7f98e4103f90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 -- 172.21.15.16:0/3083236616 >> 172.21.15.16:0/3083236616 conn(0x7f98e40ff610 msgr2=0x7f98e4101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 -- 172.21.15.16:0/3083236616 shutdown_connections 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.506+0000 7f98ea404640 1 -- 172.21.15.16:0/3083236616 wait complete. 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.507+0000 7f98ea404640 1 Processor -- start 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.507+0000 7f98ea404640 1 -- start start 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 0x7f98e419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 0x7f98e419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.565 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e4109e10 0x7f98e41a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113ad0 con 0x7f98e41094c0 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113950 con 0x7f98e4109e10 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98ea404640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98e4113c50 con 0x7f98e4103b80 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 0x7f98e419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98e3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 0x7f98e419c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37434/0 (socket says 172.21.15.16:37434) 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98e3fff640 1 -- 172.21.15.16:0/1269938857 learned_addr learned my addr 172.21.15.16:0/1269938857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98e37fe640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 0x7f98e419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.508+0000 7f98e89a1640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e4109e10 0x7f98e41a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 -- 172.21.15.16:0/1269938857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 msgr2=0x7f98e419c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 0x7f98e419c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 -- 172.21.15.16:0/1269938857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e4109e10 msgr2=0x7f98e41a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e4109e10 0x7f98e41a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 -- 172.21.15.16:0/1269938857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f98e41a2670 con 0x7f98e41094c0 2024-09-21T03:04:35.566 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e37fe640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 0x7f98e419cca0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005fd0 tx=0x7f98d800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e17fa640 1 -- 172.21.15.16:0/1269938857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8009c00 con 0x7f98e41094c0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e17fa640 1 -- 172.21.15.16:0/1269938857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d8009da0 con 0x7f98e41094c0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98e17fa640 1 -- 172.21.15.16:0/1269938857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8003e10 con 0x7f98e41094c0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98e41a2960 con 0x7f98e41094c0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.509+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98e41a90c0 con 0x7f98e41094c0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.512+0000 7f98f0978740 1 Processor -- start 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.512+0000 7f98f0978740 1 -- start start 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8b7be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4ef390 0x55dcfa8b8120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa82dea0 con 0x55dcfa510070 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa81d2d0 con 0x55dcfa4ef390 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98f0978740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa8210e0 con 0x55dcfa4e71b0 2024-09-21T03:04:35.567 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56480/0 (socket says 172.21.15.16:56480) 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 -- 172.21.15.16:0/1381147767 learned_addr learned my addr 172.21.15.16:0/1381147767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e3fff640 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8b7be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 -- 172.21.15.16:0/1381147767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4e71b0 msgr2=0x55dcfa8b7be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8b7be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 -- 172.21.15.16:0/1381147767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4ef390 msgr2=0x55dcfa8b8120 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4ef390 0x55dcfa8b8120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 -- 172.21.15.16:0/1381147767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfa57b650 con 0x55dcfa510070 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.513+0000 7f98e89a1640 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009f50 tx=0x7f98dc0118d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=621e7e36f7680c24 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.514+0000 7f98e17fa640 1 -- 172.21.15.16:0/1269938857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f98d8010460 con 0x7f98e41094c0 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.514+0000 7f98ca7fc640 1 -- 172.21.15.16:0/1381147767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc004d20 con 0x55dcfa510070 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.514+0000 7f98ca7fc640 1 -- 172.21.15.16:0/1381147767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc004050 con 0x55dcfa510070 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.514+0000 7f98ca7fc640 1 -- 172.21.15.16:0/1381147767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc005220 con 0x55dcfa510070 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.515+0000 7f98e17fa640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 0x7f98cc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.568 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.515+0000 7f98e17fa640 1 -- 172.21.15.16:0/1269938857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f98d80e7240 con 0x7f98e41094c0 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.515+0000 7f98e3fff640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 0x7f98cc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.515+0000 7f98f0978740 1 -- 172.21.15.16:0/1381147767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 msgr2=0x55dcfa8b76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.515+0000 7f98f0978740 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f98dc009f50 tx=0x7f98dc0118d0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 -- 172.21.15.16:0/1381147767 shutdown_connections 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4ef390 0x55dcfa8b8120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8b7be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 --2- 172.21.15.16:0/1381147767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8b76a0 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 -- 172.21.15.16:0/1381147767 >> 172.21.15.16:0/1381147767 conn(0x55dcfa504460 msgr2=0x55dcfa8b5590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 -- 172.21.15.16:0/1381147767 shutdown_connections 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 -- 172.21.15.16:0/1381147767 wait complete. 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.516+0000 7f98f0978740 1 -- start start 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.517+0000 7f98e3fff640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 0x7f98cc0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98d4031440 tx=0x7f98d40058d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.517+0000 7f98f0978740 1 Processor -- start 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.517+0000 7f98f0978740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c2de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.569 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.517+0000 7f98f0978740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.517+0000 7f98f0978740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98f0978740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa67e700 con 0x55dcfa510070 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98f0978740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa7b8160 con 0x55dcfa4e71b0 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98f0978740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dcfa8c5e80 con 0x55dcfa4ef390 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37454/0 (socket says 172.21.15.16:37454) 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e37fe640 1 -- 172.21.15.16:0/4089906604 learned_addr learned my addr 172.21.15.16:0/4089906604 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e3fff640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c2de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98f0978740 10 client.0 _ll_register_callbacks cb 0x55dcfa7d19e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 -- 172.21.15.16:0/4089906604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 msgr2=0x55dcfa8c3320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 -- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 msgr2=0x55dcfa8c2de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c2de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.570 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 -- 172.21.15.16:0/4089906604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfa7e5c90 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e3fff640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e37fe640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98e89a1640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8cf170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006010 tx=0x7f98dc004430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6211342726d5052 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.518+0000 7f98c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc026020 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f98dc004c50 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98dc004050 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f98dc009be0 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 10 client.15858 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dcfa8d1bd0 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98dc004050 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f98dc00f480 con 0x55dcfa510070 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 10 client.15858 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 10 client.15858 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:35.571 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 20 client.15858 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4ce9f0 0x55dcfa8d5040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 4 client.15858 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.519+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dcfa82fdf0 con 0x55dcfa4ce9f0 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.520+0000 7f98e3fff640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4ce9f0 0x55dcfa8d5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.520+0000 7f98c37fe640 10 client.15858 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.520+0000 7f98e3fff640 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4ce9f0 0x55dcfa8d5040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=695ef1f60b448770 server_cookie=a766b1cef0af0bbd in_seq=0 out_seq=0 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98c37fe640 1 -- 172.21.15.16:0/4089906604 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dcfa82fdf0 con 0x55dcfa4ce9f0 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98c37fe640 10 client.15858 handle_command_reply: tid=0 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 client.15858 shutdown 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 msgr2=0x55dcfa8cf170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8cf170 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f98dc006010 tx=0x7f98dc004430 comp rx=0 tx=0).stop 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 shutdown_connections 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4ce9f0 0x55dcfa8d5040 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa510070 0x55dcfa8cf170 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.572 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa4ef390 0x55dcfa8c3320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 --2- 172.21.15.16:0/4089906604 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c2de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 >> 172.21.15.16:0/4089906604 conn(0x55dcfa504460 msgr2=0x55dcfa8ba880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98c37fe640 0 client.15858 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 shutdown_connections 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 1 -- 172.21.15.16:0/4089906604 wait complete. 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.521+0000 7f98f0978740 20 client.15858 trim_cache size 0 max 16384 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.536+0000 7f98f0978740 1 Processor -- start 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 -- start start 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa8c0510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41af9d0 con 0x55dcfa4ef390 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b2250 con 0x55dcfa4e71b0 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98f0978740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b0040 con 0x55dcfa510070 2024-09-21T03:04:35.573 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56498/0 (socket says 172.21.15.16:56498) 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 -- 172.21.15.16:0/2567854234 learned_addr learned my addr 172.21.15.16:0/2567854234 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e89a1640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa8c0510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e3fff640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 -- 172.21.15.16:0/2567854234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 msgr2=0x55dcfa8c0510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa8c0510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 -- 172.21.15.16:0/2567854234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 msgr2=0x55dcfa8c4a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c4a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e37fe640 1 -- 172.21.15.16:0/2567854234 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dcfa8c5520 con 0x55dcfa4ef390 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.537+0000 7f98e3fff640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.538+0000 7f98e37fe640 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f98d80a9730 tx=0x7f98d80acc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6a39dcd850f9507 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.538+0000 7f98c9ffb640 1 -- 172.21.15.16:0/2567854234 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d8020030 con 0x55dcfa4ef390 2024-09-21T03:04:35.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.538+0000 7f98c9ffb640 1 -- 172.21.15.16:0/2567854234 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d80b6ad0 con 0x55dcfa4ef390 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.538+0000 7f98c9ffb640 1 -- 172.21.15.16:0/2567854234 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98d80b2d20 con 0x55dcfa4ef390 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- 172.21.15.16:0/2567854234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 msgr2=0x55dcfa8bffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f98d80a9730 tx=0x7f98d80acc70 comp rx=0 tx=0).stop 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- 172.21.15.16:0/2567854234 shutdown_connections 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8bffd0 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa8c0510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 --2- 172.21.15.16:0/2567854234 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8c4a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- 172.21.15.16:0/2567854234 >> 172.21.15.16:0/2567854234 conn(0x55dcfa504460 msgr2=0x55dcfa8c4300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- 172.21.15.16:0/2567854234 shutdown_connections 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- 172.21.15.16:0/2567854234 wait complete. 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.539+0000 7f98f0978740 1 -- start start 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 Processor -- start 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8d74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8d7a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.575 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b0710 con 0x55dcfa4ef390 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b1b80 con 0x55dcfa4e71b0 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f98e41b0de0 con 0x55dcfa510070 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98f0978740 10 client.0 _ll_register_callbacks cb 0x7f98dc009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8d7a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e89a1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e89a1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37476/0 (socket says 172.21.15.16:37476) 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e89a1640 1 -- 172.21.15.16:0/3311877189 learned_addr learned my addr 172.21.15.16:0/3311877189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8d7a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56510/0 (socket says 172.21.15.16:56510) 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.540+0000 7f98e3fff640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8d74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 -- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 msgr2=0x55dcfa8d74c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8d74c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 -- 172.21.15.16:0/3311877189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 msgr2=0x55dcfa8d7a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:35.576 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8d7a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 -- 172.21.15.16:0/3311877189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dcfa93dd00 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98e89a1640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00f920 tx=0x7f98dc009f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da26caa7e8edfcd2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98c37fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f98dc00fd20 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f98dc0044a0 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f98d80b6ad0 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f98dc0046a0 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98f0978740 10 client.25507 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55dcfa93ee90 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.541+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f98d80b6ad0 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f98dc03e400 con 0x55dcfa510070 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 10 client.25507 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 10 client.25507 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 20 client.25507 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4a3da0 0x55dcfa9415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:35.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 4 client.25507 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55dcfa82b8a0 con 0x55dcfa4a3da0 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98e3fff640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4a3da0 0x55dcfa9415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98c37fe640 10 client.25507 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.542+0000 7f98e3fff640 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4a3da0 0x55dcfa9415c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd20109953c6cc2e server_cookie=21d8bc1930e7d03e in_seq=0 out_seq=0 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98c37fe640 1 -- 172.21.15.16:0/3311877189 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55dcfa82b8a0 con 0x55dcfa4a3da0 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98c37fe640 10 client.25507 handle_command_reply: tid=0 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 client.25507 shutdown 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 msgr2=0x55dcfa93d7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f98dc00f920 tx=0x7f98dc009f60 comp rx=0 tx=0).stop 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 shutdown_connections 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dcfa4a3da0 0x55dcfa9415c0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dcfa4ef390 0x55dcfa8d7a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98c37fe640 0 client.25507 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:35.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dcfa510070 0x55dcfa93d7c0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 --2- 172.21.15.16:0/3311877189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dcfa4e71b0 0x55dcfa8d74c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 >> 172.21.15.16:0/3311877189 conn(0x55dcfa504460 msgr2=0x55dcfa8c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 shutdown_connections 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 1 -- 172.21.15.16:0/3311877189 wait complete. 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.543+0000 7f98f0978740 20 client.25507 trim_cache size 0 max 16384 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 msgr2=0x7f98cc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 0x7f98cc0b8b90 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f98d4031440 tx=0x7f98d40058d0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 msgr2=0x7f98e419cca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 0x7f98e419cca0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f98d8005fd0 tx=0x7f98d800dde0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 shutdown_connections 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f98cc0b6680 0x7f98cc0b8b90 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f98e41094c0 0x7f98e419cca0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f98e4109e10 0x7f98e41a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 --2- 172.21.15.16:0/1269938857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f98e4103b80 0x7f98e419c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 >> 172.21.15.16:0/1269938857 conn(0x7f98e40ff610 msgr2=0x7f98e410e8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 shutdown_connections 2024-09-21T03:04:35.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:35.544+0000 7f98ea404640 1 -- 172.21.15.16:0/1269938857 wait complete. 2024-09-21T03:04:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:36 smithi016 ceph-mon[31253]: pgmap v448: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 661 MiB/s wr, 1.66k op/s 2024-09-21T03:04:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:36 smithi018 ceph-mon[34587]: pgmap v448: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 661 MiB/s wr, 1.66k op/s 2024-09-21T03:04:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:36 smithi123 ceph-mon[34645]: pgmap v448: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 661 MiB/s wr, 1.66k op/s 2024-09-21T03:04:37.821 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:04:37.822 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:04:38.290 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:38 smithi016 ceph-mon[31253]: pgmap v449: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 509 MiB/s wr, 951 op/s 2024-09-21T03:04:39.162 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:38 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:04:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:38 smithi018 ceph-mon[34587]: pgmap v449: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 509 MiB/s wr, 951 op/s 2024-09-21T03:04:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:38 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:04:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:38 smithi123 ceph-mon[34645]: pgmap v449: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 509 MiB/s wr, 951 op/s 2024-09-21T03:04:39.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:38 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.321+0000 7f1229394640 1 -- 172.21.15.16:0/2902833866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 msgr2=0x7f1224102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.321+0000 7f1229394640 1 --2- 172.21.15.16:0/2902833866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f1224102e70 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1214009900 tx=0x7f121402a410 comp rx=0 tx=0).stop 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 -- 172.21.15.16:0/2902833866 shutdown_connections 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 --2- 172.21.15.16:0/2902833866 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 --2- 172.21.15.16:0/2902833866 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f12241087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 --2- 172.21.15.16:0/2902833866 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f1224102e70 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 -- 172.21.15.16:0/2902833866 >> 172.21.15.16:0/2902833866 conn(0x7f1224079730 msgr2=0x7f1224079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 -- 172.21.15.16:0/2902833866 shutdown_connections 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.322+0000 7f1229394640 1 -- 172.21.15.16:0/2902833866 wait complete. 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 Processor -- start 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 -- start start 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f12241993b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f1224199950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112960 con 0x7f1224102a60 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f12241127e0 con 0x7f1224108cf0 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.323+0000 7f1229394640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1224112ae0 con 0x7f12241083a0 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41926/0 (socket says 172.21.15.16:41926) 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 -- 172.21.15.16:0/2332968826 learned_addr learned my addr 172.21.15.16:0/2332968826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f1222ffd640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f12241993b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.344 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12227fc640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f1224199950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 -- 172.21.15.16:0/2332968826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 msgr2=0x7f1224199950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f1224199950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 -- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 msgr2=0x7f12241993b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f12241993b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12237fe640 1 -- 172.21.15.16:0/2332968826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f122419f320 con 0x7f1224108cf0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f1222ffd640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f12241993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.324+0000 7f12227fc640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f1224199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f12237fe640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f121c009fd0 tx=0x7f121c007590 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f120bfff640 1 -- 172.21.15.16:0/2332968826 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c00de10 con 0x7f1224108cf0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f120bfff640 1 -- 172.21.15.16:0/2332968826 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f121c003e20 con 0x7f1224108cf0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f120bfff640 1 -- 172.21.15.16:0/2332968826 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121c004810 con 0x7f1224108cf0 2024-09-21T03:04:39.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f122419f610 con 0x7f1224108cf0 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.325+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f122406ae60 con 0x7f1224108cf0 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.328+0000 7f122f908740 1 Processor -- start 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.328+0000 7f120bfff640 1 -- 172.21.15.16:0/2332968826 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f121c018420 con 0x7f1224108cf0 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.328+0000 7f122f908740 1 -- start start 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 0x55f92ec8b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 0x55f92ec8b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f12227fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 0x55f92ec8b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f12227fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 0x55f92ec8b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41932/0 (socket says 172.21.15.16:41932) 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92ec8beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f12227fc640 1 -- 172.21.15.16:0/696995431 learned_addr learned my addr 172.21.15.16:0/696995431 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f1222ffd640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 0x55f92ec8b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f92ec01e60 con 0x55f92e8bb490 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f92ebf1290 con 0x55f92e8e4350 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.329+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f92ebf50a0 con 0x55f92e8c3670 2024-09-21T03:04:39.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f120bfff640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 0x7f120c0b8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f12227fc640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92ec8beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f120bfff640 1 -- 172.21.15.16:0/2332968826 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f121c0e8f40 con 0x7f1224108cf0 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f1222ffd640 1 -- 172.21.15.16:0/696995431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 msgr2=0x55f92ec8beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f1222ffd640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92ec8beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f1222ffd640 1 -- 172.21.15.16:0/696995431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 msgr2=0x55f92ec8b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f1222ffd640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 0x55f92ec8b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f1222ffd640 1 -- 172.21.15.16:0/696995431 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92e94f930 con 0x55f92e8bb490 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.330+0000 7f12227fc640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92ec8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.331+0000 7f12237fe640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 0x7f120c0b8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.331+0000 7f1222ffd640 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 0x55f92ec8b970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1214009230 tx=0x7f1214032c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ce9650aca2e42ad server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.331+0000 7f12097fa640 1 -- 172.21.15.16:0/696995431 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121403f070 con 0x55f92e8bb490 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.331+0000 7f12097fa640 1 -- 172.21.15.16:0/696995431 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140316a0 con 0x55f92e8bb490 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.331+0000 7f12097fa640 1 -- 172.21.15.16:0/696995431 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214035070 con 0x55f92e8bb490 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.332+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 msgr2=0x55f92ec8b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.347 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.332+0000 7f12237fe640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 0x7f120c0b8d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f121c0aeab0 tx=0x7f121c007e50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.332+0000 7f122f908740 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 0x55f92ec8b970 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f1214009230 tx=0x7f1214032c30 comp rx=0 tx=0).stop 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 shutdown_connections 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8bb490 0x55f92ec8b970 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92ec8beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 --2- 172.21.15.16:0/696995431 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8e4350 0x55f92ec8b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 >> 172.21.15.16:0/696995431 conn(0x55f92e8d8740 msgr2=0x55f92ec89320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 shutdown_connections 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 -- 172.21.15.16:0/696995431 wait complete. 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.333+0000 7f122f908740 1 -- start start 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 Processor -- start 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8e4350 0x55f92eca1280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f92ea52890 con 0x55f92e8c3670 2024-09-21T03:04:39.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f92eb8bd40 con 0x55f92e8bb490 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f92ec9dfc0 con 0x55f92e8e4350 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f122f908740 10 client.0 _ll_register_callbacks cb 0x55f92ebefc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f1222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f12227fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f1222ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56546/0 (socket says 172.21.15.16:56546) 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f12227fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41938/0 (socket says 172.21.15.16:41938) 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f1222ffd640 1 -- 172.21.15.16:0/1286209617 learned_addr learned my addr 172.21.15.16:0/1286209617 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.334+0000 7f12227fc640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8e4350 0x55f92eca1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 -- 172.21.15.16:0/1286209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8e4350 msgr2=0x55f92eca1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8e4350 0x55f92eca1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 -- 172.21.15.16:0/1286209617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 msgr2=0x55f92ec976b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 -- 172.21.15.16:0/1286209617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92eca18f0 con 0x55f92e8bb490 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1222ffd640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:39.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f12227fc640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1218013dc0 tx=0x7f1218015040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a91a5ab8de0d9f20 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1206ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1218010e70 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f12180130d0 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12140316a0 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1218029050 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f122f908740 10 client.25556 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f92ecaa300 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.335+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12140316a0 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f121800f440 con 0x55f92e8bb490 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 10 client.25556 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 10 client.25556 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 20 client.25556 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e8a2cd0 0x55f92ecad720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 4 client.25556 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:39.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f92ec03db0 con 0x55f92e8a2cd0 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.336+0000 7f1222ffd640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e8a2cd0 0x55f92ecad720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.337+0000 7f1206ff5640 10 client.25556 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.337+0000 7f1222ffd640 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e8a2cd0 0x55f92ecad720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2d17c191b0ec1a1c server_cookie=9f4208aa53882fb0 in_seq=0 out_seq=0 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.337+0000 7f1206ff5640 1 -- 172.21.15.16:0/1286209617 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f92ec03db0 con 0x55f92e8a2cd0 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.337+0000 7f1206ff5640 10 client.25556 handle_command_reply: tid=0 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.337+0000 7f122f908740 1 client.25556 shutdown 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 msgr2=0x55f92ec970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1218013dc0 tx=0x7f1218015040 comp rx=0 tx=0).stop 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 shutdown_connections 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e8a2cd0 0x55f92ecad720 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8c3670 0x55f92ec976b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8e4350 0x55f92eca1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 --2- 172.21.15.16:0/1286209617 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec970f0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 >> 172.21.15.16:0/1286209617 conn(0x55f92e8d8740 msgr2=0x55f92ec8e610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f1206ff5640 0 client.25556 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:39.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.338+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 shutdown_connections 2024-09-21T03:04:39.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.339+0000 7f122f908740 1 -- 172.21.15.16:0/1286209617 wait complete. 2024-09-21T03:04:39.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.339+0000 7f122f908740 20 client.25556 trim_cache size 0 max 16384 2024-09-21T03:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 Processor -- start 2024-09-21T03:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 -- start start 2024-09-21T03:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8bb490 0x55f92ec906b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec940e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12241ab4c0 con 0x55f92e8e4350 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f12241add40 con 0x55f92e8c3670 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.366+0000 7f122f908740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12241abb30 con 0x55f92e8bb490 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.367+0000 7f1222ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.367+0000 7f1222ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41952/0 (socket says 172.21.15.16:41952) 2024-09-21T03:04:39.372 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.367+0000 7f1222ffd640 1 -- 172.21.15.16:0/3285657189 learned_addr learned my addr 172.21.15.16:0/3285657189 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:39.375 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f12227fc640 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec940e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f1222ffd640 1 -- 172.21.15.16:0/3285657189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8bb490 msgr2=0x55f92ec906b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f1222ffd640 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8bb490 0x55f92ec906b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f1222ffd640 1 -- 172.21.15.16:0/3285657189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 msgr2=0x55f92ec940e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f1222ffd640 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec940e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.368+0000 7f1222ffd640 1 -- 172.21.15.16:0/3285657189 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f92ec89c10 con 0x55f92e8c3670 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.369+0000 7f1222ffd640 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1214009230 tx=0x7f1214009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ceb497f5a9f9f1fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.370+0000 7f1208ff9640 1 -- 172.21.15.16:0/3285657189 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f121403f040 con 0x55f92e8c3670 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.370+0000 7f1208ff9640 1 -- 172.21.15.16:0/3285657189 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1214005dc0 con 0x55f92e8c3670 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.370+0000 7f1208ff9640 1 -- 172.21.15.16:0/3285657189 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1214035040 con 0x55f92e8c3670 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.372+0000 7f122f908740 1 -- 172.21.15.16:0/3285657189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 msgr2=0x55f92ec90bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.372+0000 7f122f908740 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1214009230 tx=0x7f1214009260 comp rx=0 tx=0).stop 2024-09-21T03:04:39.376 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 -- 172.21.15.16:0/3285657189 shutdown_connections 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec940e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8c3670 0x55f92ec90bf0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 --2- 172.21.15.16:0/3285657189 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8bb490 0x55f92ec906b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 -- 172.21.15.16:0/3285657189 >> 172.21.15.16:0/3285657189 conn(0x55f92e8d8740 msgr2=0x55f92ec95770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 -- 172.21.15.16:0/3285657189 shutdown_connections 2024-09-21T03:04:39.377 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 -- 172.21.15.16:0/3285657189 wait complete. 2024-09-21T03:04:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.374+0000 7f122f908740 1 -- start start 2024-09-21T03:04:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.375+0000 7f122f908740 1 Processor -- start 2024-09-21T03:04:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.375+0000 7f122f908740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec99970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.375+0000 7f122f908740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92eca80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.378 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f122f908740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec9a090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f122f908740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f12241ac200 con 0x55f92e8e4350 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f122f908740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f12241ad670 con 0x55f92e8bb490 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f122f908740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f12241ac8d0 con 0x55f92e8c3670 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f122f908740 10 client.0 _ll_register_callbacks cb 0x7f1218009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f12227fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec99970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f12227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec9a090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f12227fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec9a090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56554/0 (socket says 172.21.15.16:56554) 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f12227fc640 1 -- 172.21.15.16:0/4019938232 learned_addr learned my addr 172.21.15.16:0/4019938232 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:39.379 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.376+0000 7f1222ffd640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92eca80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.379+0000 7f12227fc640 1 -- 172.21.15.16:0/4019938232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 msgr2=0x55f92eca80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.379+0000 7f12227fc640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92eca80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.379+0000 7f12227fc640 1 -- 172.21.15.16:0/4019938232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 msgr2=0x55f92ec9a090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.379+0000 7f12227fc640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec9a090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.382 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.379+0000 7f12227fc640 1 -- 172.21.15.16:0/4019938232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f92eca30e0 con 0x55f92e8bb490 2024-09-21T03:04:39.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.381+0000 7f12227fc640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec99970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1218001eb0 tx=0x7f1218005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=186624c6d258d134 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:39.383 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.382+0000 7f1206ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:39.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.382+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12180130d0 con 0x55f92e8bb490 2024-09-21T03:04:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.382+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1214005dc0 con 0x55f92e8bb490 2024-09-21T03:04:39.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.383+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f121800f440 con 0x55f92e8bb490 2024-09-21T03:04:39.386 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.384+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f121800f640 con 0x55f92e8bb490 2024-09-21T03:04:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.385+0000 7f122f908740 10 client.25562 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.385+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f92ed144b0 con 0x55f92e8bb490 2024-09-21T03:04:39.387 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.385+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1214005dc0 con 0x55f92e8bb490 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.385+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1218022990 con 0x55f92e8bb490 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 10 client.25562 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 10 client.25562 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 20 client.25562 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e878080 0x55f92ed16c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 4 client.25562 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:04:39.388 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.386+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f92ebff860 con 0x55f92e878080 2024-09-21T03:04:39.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.393+0000 7f1222ffd640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e878080 0x55f92ed16c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:39.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.397+0000 7f1222ffd640 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e878080 0x55f92ed16c30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a9a39fb3c407bf95 server_cookie=49865175fc844020 in_seq=0 out_seq=0 2024-09-21T03:04:39.399 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.397+0000 7f1206ff5640 10 client.25562 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f1206ff5640 1 -- 172.21.15.16:0/4019938232 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f92ebff860 con 0x55f92e878080 2024-09-21T03:04:39.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f1206ff5640 10 client.25562 handle_command_reply: tid=0 2024-09-21T03:04:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f122f908740 1 client.25562 shutdown 2024-09-21T03:04:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 msgr2=0x55f92ec99970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec99970 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1218001eb0 tx=0x7f1218005f90 comp rx=0 tx=0).stop 2024-09-21T03:04:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.399+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 shutdown_connections 2024-09-21T03:04:39.401 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f92e878080 0x55f92ed16c30 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f92e8e4350 0x55f92ec9a090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f92e8c3670 0x55f92eca80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 --2- 172.21.15.16:0/4019938232 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f92e8bb490 0x55f92ec99970 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 >> 172.21.15.16:0/4019938232 conn(0x55f92e8d8740 msgr2=0x55f92ec983a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f1206ff5640 0 client.25562 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:39.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 shutdown_connections 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 1 -- 172.21.15.16:0/4019938232 wait complete. 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.400+0000 7f122f908740 20 client.25562 trim_cache size 0 max 16384 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "3b241a9e-e2ba-4add-bf02-6d772b7c0821", 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:04:39.403 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 msgr2=0x7f120c0b8d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 0x7f120c0b8d30 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f121c0aeab0 tx=0x7f121c007e50 comp rx=0 tx=0).stop 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 msgr2=0x7f122419ebf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f121c009fd0 tx=0x7f121c007590 comp rx=0 tx=0).stop 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 shutdown_connections 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f120c0b6820 0x7f120c0b8d30 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1224108cf0 0x7f122419ebf0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12241083a0 0x7f1224199950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 --2- 172.21.15.16:0/2332968826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1224102a60 0x7f12241993b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 >> 172.21.15.16:0/2332968826 conn(0x7f1224079730 msgr2=0x7f1224101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 shutdown_connections 2024-09-21T03:04:39.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:39.401+0000 7f1229394640 1 -- 172.21.15.16:0/2332968826 wait complete. 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:40.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:39 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:04:40.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:40.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:39 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:40.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:39 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:04:40.783 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:04:41.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:40 smithi016 ceph-mon[31253]: pgmap v450: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 464 MiB/s wr, 886 op/s 2024-09-21T03:04:41.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:40 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:40 smithi018 ceph-mon[34587]: pgmap v450: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 464 MiB/s wr, 886 op/s 2024-09-21T03:04:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:40 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:41.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:40 smithi123 ceph-mon[34645]: pgmap v450: 97 pgs: 97 active+clean; 39 GiB data, 118 GiB used, 955 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 464 MiB/s wr, 886 op/s 2024-09-21T03:04:41.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:40 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:41.558 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 -- 172.21.15.16:0/3669008838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 msgr2=0x7f995c1106f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 --2- 172.21.15.16:0/3669008838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1106f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9954009ca0 tx=0x7f995402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 -- 172.21.15.16:0/3669008838 shutdown_connections 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 --2- 172.21.15.16:0/3669008838 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1106f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 --2- 172.21.15.16:0/3669008838 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c109980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 --2- 172.21.15.16:0/3669008838 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 -- 172.21.15.16:0/3669008838 >> 172.21.15.16:0/3669008838 conn(0x7f995c07b8d0 msgr2=0x7f995c07bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 -- 172.21.15.16:0/3669008838 shutdown_connections 2024-09-21T03:04:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.761+0000 7f996132d640 1 -- 172.21.15.16:0/3669008838 wait complete. 2024-09-21T03:04:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.762+0000 7f996132d640 1 Processor -- start 2024-09-21T03:04:42.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.762+0000 7f996132d640 1 -- start start 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c1a1ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1ac820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f995c113ae0 con 0x7f995c109ec0 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f995c113960 con 0x7f995c109570 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f996132d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f995c113c60 con 0x7f995c103c30 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f995affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f995affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37560/0 (socket says 172.21.15.16:37560) 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.763+0000 7f995affd640 1 -- 172.21.15.16:0/276106043 learned_addr learned my addr 172.21.15.16:0/276106043 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:42.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995b7fe640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1ac820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 -- 172.21.15.16:0/276106043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 msgr2=0x7f995c1a1ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f9953fff640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c1a1ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c1a1ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 -- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 msgr2=0x7f995c1ac820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1ac820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 -- 172.21.15.16:0/276106043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f995c1a2560 con 0x7f995c103c30 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995b7fe640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f9953fff640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.764+0000 7f995affd640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00eeb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.765+0000 7f9958ff9640 1 -- 172.21.15.16:0/276106043 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00cd80 con 0x7f995c103c30 2024-09-21T03:04:42.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.765+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f995c1ace20 con 0x7f995c103c30 2024-09-21T03:04:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.765+0000 7f9958ff9640 1 -- 172.21.15.16:0/276106043 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f994c0108e0 con 0x7f995c103c30 2024-09-21T03:04:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.765+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f995c1ad320 con 0x7f995c103c30 2024-09-21T03:04:42.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.765+0000 7f9958ff9640 1 -- 172.21.15.16:0/276106043 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994c00cd80 con 0x7f995c103c30 2024-09-21T03:04:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.769+0000 7f99678a1740 1 Processor -- start 2024-09-21T03:04:42.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 -- start start 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb14867b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1486cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fb13fce50 con 0x555fb10b61b0 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555fb13ec280 con 0x555fb10be390 2024-09-21T03:04:42.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.770+0000 7f99678a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555fb13f0090 con 0x555fb10df070 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37566/0 (socket says 172.21.15.16:37566) 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 -- 172.21.15.16:0/1814831832 learned_addr learned my addr 172.21.15.16:0/1814831832 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f9958ff9640 1 -- 172.21.15.16:0/276106043 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f994c018760 con 0x7f995c103c30 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995affd640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb14867b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f9953fff640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1486cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 -- 172.21.15.16:0/1814831832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 msgr2=0x555fb1486cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1486cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 -- 172.21.15.16:0/1814831832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 msgr2=0x555fb14867b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb14867b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995b7fe640 1 -- 172.21.15.16:0/1814831832 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb114a650 con 0x555fb10df070 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f9953fff640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1486cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.771+0000 7f995affd640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb14867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:42.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.772+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9954009ca0 tx=0x7f9954009460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=aecd270082454823 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.772+0000 7f99517fa640 1 -- 172.21.15.16:0/1814831832 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954031cf0 con 0x555fb10df070 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.772+0000 7f9958ff9640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 0x7f99480b8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.772+0000 7f99517fa640 1 -- 172.21.15.16:0/1814831832 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9954036040 con 0x555fb10df070 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.772+0000 7f99517fa640 1 -- 172.21.15.16:0/1814831832 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540042a0 con 0x555fb10df070 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.773+0000 7f9958ff9640 1 -- 172.21.15.16:0/276106043 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f994c00cb80 con 0x7f995c103c30 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.773+0000 7f995affd640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 0x7f99480b8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.773+0000 7f99678a1740 1 -- 172.21.15.16:0/1814831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 msgr2=0x555fb1486270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.773+0000 7f99678a1740 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f9954009ca0 tx=0x7f9954009460 comp rx=0 tx=0).stop 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f995affd640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 0x7f99480b8fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555fb1484ef0 tx=0x7f994c018420 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 -- 172.21.15.16:0/1814831832 shutdown_connections 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb14867b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1486cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 --2- 172.21.15.16:0/1814831832 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10df070 0x555fb1486270 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 -- 172.21.15.16:0/1814831832 >> 172.21.15.16:0/1814831832 conn(0x555fb10d3460 msgr2=0x555fb1484160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 -- 172.21.15.16:0/1814831832 shutdown_connections 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 -- 172.21.15.16:0/1814831832 wait complete. 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.774+0000 7f99678a1740 1 -- start start 2024-09-21T03:04:42.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 Processor -- start 2024-09-21T03:04:42.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 0x555fb1491980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10df070 0x555fb149dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555fb124d700 con 0x555fb10df070 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555fb1387110 con 0x555fb10be390 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555fb14949f0 con 0x555fb10b61b0 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f99678a1740 10 client.0 _ll_register_callbacks cb 0x555fb13a0990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f9953fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 0x555fb1491980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f9953fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:41992/0 (socket says 172.21.15.16:41992) 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f9953fff640 1 -- 172.21.15.16:0/1892677084 learned_addr learned my addr 172.21.15.16:0/1892677084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.775+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10df070 0x555fb149dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 -- 172.21.15.16:0/1892677084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 msgr2=0x555fb1491980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 0x555fb1491980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 -- 172.21.15.16:0/1892677084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10df070 msgr2=0x555fb149dce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10df070 0x555fb149dce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 -- 172.21.15.16:0/1892677084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb13b4c40 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 0x555fb1491980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f9953fff640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f994400de50 tx=0x7f994400f730 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3b14eb60c87f6cf1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f99427fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f994400fea0 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.776+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9944004220 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9954036040 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9944004420 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 10 client.25568 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555fb14a0730 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9954036040 con 0x555fb10be390 2024-09-21T03:04:42.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f9944003990 con 0x555fb10be390 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 10 client.25568 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 10 client.25568 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 20 client.25568 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.777+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb109d9f0 0x555fb14a3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.778+0000 7f99678a1740 4 client.25568 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.778+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fb13feda0 con 0x555fb109d9f0 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.778+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb109d9f0 0x555fb14a3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.778+0000 7f99427fc640 10 client.25568 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:42.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.778+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb109d9f0 0x555fb14a3c80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2cde79e6d0b12319 server_cookie=5eded23771ffe7f in_seq=0 out_seq=0 2024-09-21T03:04:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.779+0000 7f99427fc640 1 -- 172.21.15.16:0/1892677084 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x555fb13feda0 con 0x555fb109d9f0 2024-09-21T03:04:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.779+0000 7f99427fc640 10 client.25568 handle_command_reply: tid=0 2024-09-21T03:04:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.779+0000 7f99678a1740 1 client.25568 shutdown 2024-09-21T03:04:42.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.779+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 msgr2=0x555fb1491ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.779+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f994400de50 tx=0x7f994400f730 comp rx=0 tx=0).stop 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 shutdown_connections 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb109d9f0 0x555fb14a3c80 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99427fc640 0 client.25568 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10df070 0x555fb149dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1491ec0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 --2- 172.21.15.16:0/1892677084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb10b61b0 0x555fb1491980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 >> 172.21.15.16:0/1892677084 conn(0x555fb10d3460 msgr2=0x555fb1489400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 shutdown_connections 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 1 -- 172.21.15.16:0/1892677084 wait complete. 2024-09-21T03:04:42.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.780+0000 7f99678a1740 20 client.25568 trim_cache size 0 max 16384 2024-09-21T03:04:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 Processor -- start 2024-09-21T03:04:42.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 -- start start 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb1491f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10b61b0 0x555fb1484190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b3e40 con 0x555fb10be390 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b66c0 con 0x555fb10b61b0 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f99678a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b44b0 con 0x555fb109d9f0 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f9953fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10b61b0 0x555fb1484190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:56602/0 (socket says 172.21.15.16:56602) 2024-09-21T03:04:42.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f995b7fe640 1 -- 172.21.15.16:0/1627145011 learned_addr learned my addr 172.21.15.16:0/1627145011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.823+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb1491f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 -- 172.21.15.16:0/1627145011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 msgr2=0x555fb1491f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb1491f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 -- 172.21.15.16:0/1627145011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10b61b0 msgr2=0x555fb1484190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10b61b0 0x555fb1484190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 -- 172.21.15.16:0/1627145011 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fb14a41f0 con 0x555fb10be390 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9954003d40 tx=0x7f9954005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4e3885b58c5b34 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f9950ff9640 1 -- 172.21.15.16:0/1627145011 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f995404a040 con 0x555fb10be390 2024-09-21T03:04:42.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f9950ff9640 1 -- 172.21.15.16:0/1627145011 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9954036030 con 0x555fb10be390 2024-09-21T03:04:42.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.824+0000 7f9950ff9640 1 -- 172.21.15.16:0/1627145011 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f99540047f0 con 0x555fb10be390 2024-09-21T03:04:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 -- 172.21.15.16:0/1627145011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 msgr2=0x555fb149dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9954003d40 tx=0x7f9954005d80 comp rx=0 tx=0).stop 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 -- 172.21.15.16:0/1627145011 shutdown_connections 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10be390 0x555fb149dbe0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10b61b0 0x555fb1484190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 --2- 172.21.15.16:0/1627145011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb1491f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 -- 172.21.15.16:0/1627145011 >> 172.21.15.16:0/1627145011 conn(0x555fb10d3460 msgr2=0x555fb1483a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 -- 172.21.15.16:0/1627145011 shutdown_connections 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.825+0000 7f99678a1740 1 -- 172.21.15.16:0/1627145011 wait complete. 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 -- start start 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 Processor -- start 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb149a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb1495970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b4b80 con 0x555fb10b61b0 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b5ff0 con 0x555fb10be390 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f995c1b5250 con 0x555fb109d9f0 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.826+0000 7f99678a1740 10 client.0 _ll_register_callbacks cb 0x7f994400d380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:42016/0 (socket says 172.21.15.16:42016) 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 -- 172.21.15.16:0/1538073702 learned_addr learned my addr 172.21.15.16:0/1538073702 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb149a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 -- 172.21.15.16:0/1538073702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 msgr2=0x555fb149a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb149a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 -- 172.21.15.16:0/1538073702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 msgr2=0x555fb1495970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb1495970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 -- 172.21.15.16:0/1538073702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fb14966d0 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f995b7fe640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9954009bf0 tx=0x7f9954004490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b72f03da02b44dc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.827+0000 7f99427fc640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9954004570 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9954004710 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9954036030 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9954031450 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 10 client.25574 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555fb150e600 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9954036030 con 0x555fb10be390 2024-09-21T03:04:42.833 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f995402fb70 con 0x555fb10be390 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 10 client.25574 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 10 client.25574 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 20 client.25574 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb10df070 0x555fb15107e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 4 client.25574 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.828+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fb13fa850 con 0x555fb10df070 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.829+0000 7f9953fff640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb10df070 0x555fb15107e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.829+0000 7f99427fc640 10 client.25574 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:42.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.829+0000 7f9953fff640 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb10df070 0x555fb15107e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c20515fe2fbcaced server_cookie=d1704e178270009d in_seq=0 out_seq=0 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99427fc640 1 -- 172.21.15.16:0/1538073702 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2362 (crc 0 0 0) 0x555fb13fa850 con 0x555fb10df070 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99427fc640 10 client.25574 handle_command_reply: tid=0 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 client.25574 shutdown 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 msgr2=0x555fb1495f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f9954009bf0 tx=0x7f9954004490 comp rx=0 tx=0).stop 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 shutdown_connections 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555fb10df070 0x555fb15107e0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555fb10b61b0 0x555fb1495970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555fb10be390 0x555fb1495f70 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99427fc640 0 client.25574 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:42.836 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 --2- 172.21.15.16:0/1538073702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555fb109d9f0 0x555fb149a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 >> 172.21.15.16:0/1538073702 conn(0x555fb10d3460 msgr2=0x555fb1492d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 shutdown_connections 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 1 -- 172.21.15.16:0/1538073702 wait complete. 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.830+0000 7f99678a1740 20 client.25574 trim_cache size 0 max 16384 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:04:42.837 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.839 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.840 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.841 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:42.842 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:04:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.831+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 msgr2=0x7f99480b8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.831+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 0x7f99480b8fe0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x555fb1484ef0 tx=0x7f994c018420 comp rx=0 tx=0).stop 2024-09-21T03:04:42.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.831+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 msgr2=0x7f995c1a17a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.831+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f994c00e9f0 tx=0x7f994c00eeb0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 shutdown_connections 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f99480b6ad0 0x7f99480b8fe0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f995c109ec0 0x7f995c1ac820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f995c109570 0x7f995c1a1ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 --2- 172.21.15.16:0/276106043 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f995c103c30 0x7f995c1a17a0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 >> 172.21.15.16:0/276106043 conn(0x7f995c07b8d0 msgr2=0x7f995c1073e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 shutdown_connections 2024-09-21T03:04:42.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:42.832+0000 7f996132d640 1 -- 172.21.15.16:0/276106043 wait complete. 2024-09-21T03:04:43.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:43 smithi016 ceph-mon[31253]: pgmap v451: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 538 MiB/s wr, 1.25k op/s 2024-09-21T03:04:43.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:43 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:04:43.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:43 smithi123 ceph-mon[34645]: pgmap v451: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 538 MiB/s wr, 1.25k op/s 2024-09-21T03:04:43.272 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:43 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:04:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:43 smithi018 ceph-mon[34587]: pgmap v451: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 538 MiB/s wr, 1.25k op/s 2024-09-21T03:04:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:43 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:04:44.199 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:04:44.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:04:44.688 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:45.101 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:45 smithi016 ceph-mon[31253]: pgmap v452: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 376 MiB/s wr, 929 op/s 2024-09-21T03:04:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:45 smithi018 ceph-mon[34587]: pgmap v452: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 376 MiB/s wr, 929 op/s 2024-09-21T03:04:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:45 smithi123 ceph-mon[34645]: pgmap v452: 97 pgs: 97 active+clean; 39 GiB data, 119 GiB used, 954 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 376 MiB/s wr, 929 op/s 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 -- 172.21.15.16:0/2313216684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 msgr2=0x7f083c11f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 --2- 172.21.15.16:0/2313216684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c11f140 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f082c009900 tx=0x7f082c02a410 comp rx=0 tx=0).stop 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 -- 172.21.15.16:0/2313216684 shutdown_connections 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 --2- 172.21.15.16:0/2313216684 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c11f140 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 --2- 172.21.15.16:0/2313216684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 0x7f083c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:45.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 --2- 172.21.15.16:0/2313216684 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 0x7f083c10a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.575+0000 7f08424ff640 1 -- 172.21.15.16:0/2313216684 >> 172.21.15.16:0/2313216684 conn(0x7f083c070ec0 msgr2=0x7f083c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.576+0000 7f08424ff640 1 -- 172.21.15.16:0/2313216684 shutdown_connections 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.576+0000 7f08424ff640 1 -- 172.21.15.16:0/2313216684 wait complete. 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.577+0000 7f08424ff640 1 Processor -- start 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.577+0000 7f08424ff640 1 -- start start 2024-09-21T03:04:45.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 0x7f083c112670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 0x7f083c112bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f083c122910 con 0x7f083c11cc30 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f083c122790 con 0x7f083c111f40 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f08424ff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f083c122a90 con 0x7f083c10a9e0 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f0840a9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f0840a9c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42920/0 (socket says 172.21.15.16:42920) 2024-09-21T03:04:45.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.578+0000 7f0840a9c640 1 -- 172.21.15.16:0/1189556469 learned_addr learned my addr 172.21.15.16:0/1189556469 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.579+0000 7f083bfff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 0x7f083c112670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.579+0000 7f083b7fe640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 0x7f083c112bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.580+0000 7f0840a9c640 1 -- 172.21.15.16:0/1189556469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 msgr2=0x7f083c112670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:45.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.580+0000 7f0840a9c640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 0x7f083c112670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.580+0000 7f0840a9c640 1 -- 172.21.15.16:0/1189556469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 msgr2=0x7f083c112bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.580+0000 7f0840a9c640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 0x7f083c112bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:45.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.580+0000 7f0840a9c640 1 -- 172.21.15.16:0/1189556469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f083c1185e0 con 0x7f083c11cc30 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f0840a9c640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f082c02a980 tx=0x7f082c003c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c03f070 con 0x7f083c11cc30 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f082c031b50 con 0x7f083c11cc30 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f082c035030 con 0x7f083c11cc30 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f083c118810 con 0x7f083c11cc30 2024-09-21T03:04:45.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.581+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f083c119020 con 0x7f083c11cc30 2024-09-21T03:04:45.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.583+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f083c10b5e0 con 0x7f083c11cc30 2024-09-21T03:04:45.592 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.590+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f082c042070 con 0x7f083c11cc30 2024-09-21T03:04:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.591+0000 7f08397fa640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 0x7f08240b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:45.593 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.592+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f082c108670 con 0x7f083c11cc30 2024-09-21T03:04:45.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.596+0000 7f083bfff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 0x7f08240b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:45.598 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.597+0000 7f083bfff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 0x7f08240b8bc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f083c118f40 tx=0x7f0834009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:45.600 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:45.598+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f082c0d2fc0 con 0x7f083c11cc30 2024-09-21T03:04:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.064+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f083c1191f0 con 0x7f083c11cc30 2024-09-21T03:04:46.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.065+0000 7f08397fa640 1 -- 172.21.15.16:0/1189556469 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f082c0d7f00 con 0x7f083c11cc30 2024-09-21T03:04:46.066 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:04:46.067 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:04:46.067 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.067+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 msgr2=0x7f08240b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.067+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 0x7f08240b8bc0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f083c118f40 tx=0x7f0834009500 comp rx=0 tx=0).stop 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.067+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 msgr2=0x7f083c117eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.067+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f082c02a980 tx=0x7f082c003c00 comp rx=0 tx=0).stop 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 shutdown_connections 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f08240b66b0 0x7f08240b8bc0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:46.069 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f083c11cc30 0x7f083c117eb0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f083c111f40 0x7f083c112bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 --2- 172.21.15.16:0/1189556469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f083c10a9e0 0x7f083c112670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 >> 172.21.15.16:0/1189556469 conn(0x7f083c070ec0 msgr2=0x7f083c11e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 shutdown_connections 2024-09-21T03:04:46.070 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:46.068+0000 7f08424ff640 1 -- 172.21.15.16:0/1189556469 wait complete. 2024-09-21T03:04:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:46 smithi018 ceph-mon[34587]: pgmap v453: 97 pgs: 97 active+clean; 40 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 527 MiB/s wr, 990 op/s 2024-09-21T03:04:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:46 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1189556469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:46.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:46 smithi123 ceph-mon[34645]: pgmap v453: 97 pgs: 97 active+clean; 40 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 527 MiB/s wr, 990 op/s 2024-09-21T03:04:46.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:46 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1189556469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:46 smithi016 ceph-mon[31253]: pgmap v453: 97 pgs: 97 active+clean; 40 GiB data, 121 GiB used, 951 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 527 MiB/s wr, 990 op/s 2024-09-21T03:04:46.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:46 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1189556469' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:04:47.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:04:48.206 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:48.714 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:48 smithi016 ceph-mon[31253]: pgmap v454: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 460 MiB/s wr, 917 op/s 2024-09-21T03:04:48.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:48 smithi123 ceph-mon[34645]: pgmap v454: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 460 MiB/s wr, 917 op/s 2024-09-21T03:04:48.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:48 smithi018 ceph-mon[34587]: pgmap v454: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 460 MiB/s wr, 917 op/s 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 -- 172.21.15.16:0/2317195054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00a53a0 msgr2=0x7f01c00a57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 --2- 172.21.15.16:0/2317195054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00a53a0 0x7f01c00a57b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01c8066480 tx=0x7f01c8090cd0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 -- 172.21.15.16:0/2317195054 shutdown_connections 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 --2- 172.21.15.16:0/2317195054 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00aace0 0x7f01c00ab0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 --2- 172.21.15.16:0/2317195054 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00ab630 0x7f01c00b1e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 --2- 172.21.15.16:0/2317195054 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00a53a0 0x7f01c00a57b0 secure :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f01c8066480 tx=0x7f01c8090cd0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.262 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 -- 172.21.15.16:0/2317195054 >> 172.21.15.16:0/2317195054 conn(0x7f01c001d0d0 msgr2=0x7f01c00a39f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.260+0000 7f01cd631640 1 -- 172.21.15.16:0/2317195054 shutdown_connections 2024-09-21T03:04:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.261+0000 7f01cd631640 1 -- 172.21.15.16:0/2317195054 wait complete. 2024-09-21T03:04:49.263 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.262+0000 7f01cd631640 1 Processor -- start 2024-09-21T03:04:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.262+0000 7f01cd631640 1 -- start start 2024-09-21T03:04:49.264 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00aace0 0x7f01c00b6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00bbfd0 0x7f01c00bc3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00b4ba0 con 0x7f01c00ab630 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00b4a20 con 0x7f01c00aace0 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.263+0000 7f01cd631640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01c00b4d20 con 0x7f01c00bbfd0 2024-09-21T03:04:49.265 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.264+0000 7f01c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.264+0000 7f01c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42930/0 (socket says 172.21.15.16:42930) 2024-09-21T03:04:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.264+0000 7f01c67fc640 1 -- 172.21.15.16:0/575780857 learned_addr learned my addr 172.21.15.16:0/575780857 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.264+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00aace0 0x7f01c00b6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.266 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c77fe640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00bbfd0 0x7f01c00bc3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c67fc640 1 -- 172.21.15.16:0/575780857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00bbfd0 msgr2=0x7f01c00bc3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c67fc640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00bbfd0 0x7f01c00bc3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c67fc640 1 -- 172.21.15.16:0/575780857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00aace0 msgr2=0x7f01c00b6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c67fc640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00aace0 0x7f01c00b6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.267 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.265+0000 7f01c67fc640 1 -- 172.21.15.16:0/575780857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01c00bcc00 con 0x7f01c00ab630 2024-09-21T03:04:49.271 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.269+0000 7f01c67fc640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006fd0 tx=0x7f01b800b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.274+0000 7f01affff640 1 -- 172.21.15.16:0/575780857 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8011830 con 0x7f01c00ab630 2024-09-21T03:04:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.274+0000 7f01affff640 1 -- 172.21.15.16:0/575780857 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b8016070 con 0x7f01c00ab630 2024-09-21T03:04:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.274+0000 7f01affff640 1 -- 172.21.15.16:0/575780857 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b8010e00 con 0x7f01c00ab630 2024-09-21T03:04:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.274+0000 7f01cd631640 1 -- 172.21.15.16:0/575780857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01c00c1160 con 0x7f01c00ab630 2024-09-21T03:04:49.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.274+0000 7f01cd631640 1 -- 172.21.15.16:0/575780857 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01c00c1620 con 0x7f01c00ab630 2024-09-21T03:04:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.284+0000 7f01d3ba5740 1 Processor -- start 2024-09-21T03:04:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 -- start start 2024-09-21T03:04:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 0x55b601669170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601666780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 0x55b601666d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b6015dcea0 con 0x55b6012bf070 2024-09-21T03:04:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b6015cc2d0 con 0x55b60129e390 2024-09-21T03:04:49.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.285+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b6015d00e0 con 0x55b6012961b0 2024-09-21T03:04:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.298+0000 7f01c67fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 0x55b601666d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.298+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601666780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.298+0000 7f01c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 0x55b601669170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.298+0000 7f01c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 0x55b601669170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42938/0 (socket says 172.21.15.16:42938) 2024-09-21T03:04:49.301 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.298+0000 7f01c77fe640 1 -- 172.21.15.16:0/3794823318 learned_addr learned my addr 172.21.15.16:0/3794823318 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.299+0000 7f01c67fc640 1 -- 172.21.15.16:0/3794823318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 msgr2=0x55b601666780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.299+0000 7f01c67fc640 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601666780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.300+0000 7f01c67fc640 1 -- 172.21.15.16:0/3794823318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 msgr2=0x55b601669170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.300+0000 7f01c67fc640 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 0x55b601669170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.302 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.300+0000 7f01c67fc640 1 -- 172.21.15.16:0/3794823318 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b60132a650 con 0x55b60129e390 2024-09-21T03:04:49.305 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.303+0000 7f01affff640 1 -- 172.21.15.16:0/575780857 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f01b80253d0 con 0x7f01c00ab630 2024-09-21T03:04:49.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.304+0000 7f01affff640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 0x7f01b00b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.305+0000 7f01affff640 1 -- 172.21.15.16:0/575780857 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01b80e8560 con 0x7f01c00ab630 2024-09-21T03:04:49.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.306+0000 7f01c67fc640 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 0x55b601666d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01c00c17f0 tx=0x7f01b8011de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bb380b3f4bf78c16 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.314 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.307+0000 7f01cd631640 1 -- 172.21.15.16:0/3794823318 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80aa430 con 0x55b60129e390 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.307+0000 7f01cd631640 1 -- 172.21.15.16:0/3794823318 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01b80ae0a0 con 0x55b60129e390 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.308+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3794823318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 msgr2=0x55b601666d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.308+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 0x55b601666d20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f01c00c17f0 tx=0x7f01b8011de0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3794823318 shutdown_connections 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b60129e390 0x55b601666d20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601666780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3794823318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012bf070 0x55b601669170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3794823318 >> 172.21.15.16:0/3794823318 conn(0x55b6012b3460 msgr2=0x55b601664670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3794823318 shutdown_connections 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.309+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3794823318 wait complete. 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.310+0000 7f01d3ba5740 1 -- start start 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.310+0000 7f01d3ba5740 1 Processor -- start 2024-09-21T03:04:49.315 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.310+0000 7f01c77fe640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 0x7f01b00b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.316 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601674500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b6016719d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55b60142d700 con 0x55b60127d9f0 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55b601567160 con 0x55b6012bf070 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55b601674c20 con 0x55b6012961b0 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.313+0000 7f01d3ba5740 10 client.0 _ll_register_callbacks cb 0x55b6015809e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.314+0000 7f01c77fe640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 0x7f01b00b8d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f01bc005fd0 tx=0x7f01bc005f60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.314+0000 7f01c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.314+0000 7f01c77fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42946/0 (socket says 172.21.15.16:42946) 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.314+0000 7f01c77fe640 1 -- 172.21.15.16:0/3968965852 learned_addr learned my addr 172.21.15.16:0/3968965852 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:49.317 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601674500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c77fe640 1 -- 172.21.15.16:0/3968965852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 msgr2=0x55b601674500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c77fe640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601674500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c77fe640 1 -- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 msgr2=0x55b6016719d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c77fe640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b6016719d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.315+0000 7f01c77fe640 1 -- 172.21.15.16:0/3968965852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b601594c90 con 0x55b60127d9f0 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.316+0000 7f01c77fe640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f01bc01c850 tx=0x7f01bc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5784f0a853bffde server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.318 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.316+0000 7f01aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.317+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01bc02c060 con 0x55b60127d9f0 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.318+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6015dedf0 con 0x55b60127d9f0 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.318+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01bc016030 con 0x55b60127d9f0 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.322+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f01bc027420 con 0x55b60127d9f0 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.322+0000 7f01d3ba5740 10 client.15909 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:49.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.322+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6016805f0 con 0x55b60127d9f0 2024-09-21T03:04:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6015dedf0 con 0x55b60127d9f0 2024-09-21T03:04:49.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f01bc00f790 con 0x55b60127d9f0 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 10 client.15909 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 10 client.15909 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 20 client.15909 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b601252da0 0x55b601683fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 4 client.15909 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:49.329 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.327+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6015da8a0 con 0x55b601252da0 2024-09-21T03:04:49.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.329+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b601252da0 0x55b601683fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.345 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.339+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b601252da0 0x55b601683fa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b99350e786fea4f6 server_cookie=214da01fd9610d10 in_seq=0 out_seq=0 2024-09-21T03:04:49.346 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.339+0000 7f01aaff5640 10 client.15909 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.344+0000 7f01aaff5640 1 -- 172.21.15.16:0/3968965852 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55b6015da8a0 con 0x55b601252da0 2024-09-21T03:04:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.344+0000 7f01aaff5640 10 client.15909 handle_command_reply: tid=0 2024-09-21T03:04:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.344+0000 7f01d3ba5740 1 client.15909 shutdown 2024-09-21T03:04:49.349 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 msgr2=0x55b6016720a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f01bc01c850 tx=0x7f01bc004730 comp rx=0 tx=0).stop 2024-09-21T03:04:49.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 shutdown_connections 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b601252da0 0x55b601683fa0 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b60127d9f0 0x55b6016720a0 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b6016719d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/3968965852 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b6012961b0 0x55b601674500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.346+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 >> 172.21.15.16:0/3968965852 conn(0x55b6012b3460 msgr2=0x55b601668170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.347+0000 7f01aaff5640 0 client.15909 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.347+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 shutdown_connections 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.347+0000 7f01d3ba5740 1 -- 172.21.15.16:0/3968965852 wait complete. 2024-09-21T03:04:49.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.347+0000 7f01d3ba5740 20 client.15909 trim_cache size 0 max 16384 2024-09-21T03:04:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.381+0000 7f01d3ba5740 1 Processor -- start 2024-09-21T03:04:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.381+0000 7f01d3ba5740 1 -- start start 2024-09-21T03:04:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.382+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.384 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.383+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016840e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.383+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.385 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.383+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52618/0 (socket says 172.21.15.16:52618) 2024-09-21T03:04:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.383+0000 7f01c6ffd640 1 -- 172.21.15.16:0/2345712090 learned_addr learned my addr 172.21.15.16:0/2345712090 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.384+0000 7f01c67fc640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016840e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.391 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.384+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601666280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.384+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01c015fe50 con 0x55b6012961b0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.384+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01626d0 con 0x55b6012bf070 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.384+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01c01604c0 con 0x55b60127d9f0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.385+0000 7f01c77fe640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601666280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.385+0000 7f01c6ffd640 1 -- 172.21.15.16:0/2345712090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 msgr2=0x55b601666280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.385+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601666280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.385+0000 7f01c6ffd640 1 -- 172.21.15.16:0/2345712090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 msgr2=0x55b6016840e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.386+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016840e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.386+0000 7f01c6ffd640 1 -- 172.21.15.16:0/2345712090 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6016717a0 con 0x55b60127d9f0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.386+0000 7f01c67fc640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.387+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01c8052910 tx=0x7f01c8069570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cba71b8ce16a7f25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.388+0000 7f01acff9640 1 -- 172.21.15.16:0/2345712090 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8063d20 con 0x55b60127d9f0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.388+0000 7f01acff9640 1 -- 172.21.15.16:0/2345712090 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8095040 con 0x55b60127d9f0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.388+0000 7f01acff9640 1 -- 172.21.15.16:0/2345712090 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01c8091a30 con 0x55b60127d9f0 2024-09-21T03:04:49.392 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.389+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 msgr2=0x55b60167de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.389+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f01c8052910 tx=0x7f01c8069570 comp rx=0 tx=0).stop 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 shutdown_connections 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016840e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601666280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/2345712090 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b60167de50 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.393 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 >> 172.21.15.16:0/2345712090 conn(0x55b6012b3460 msgr2=0x55b601670270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 shutdown_connections 2024-09-21T03:04:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.390+0000 7f01d3ba5740 1 -- 172.21.15.16:0/2345712090 wait complete. 2024-09-21T03:04:49.394 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.391+0000 7f01d3ba5740 1 -- start start 2024-09-21T03:04:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.392+0000 7f01d3ba5740 1 Processor -- start 2024-09-21T03:04:49.395 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b6016e6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01d3ba5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601679cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b6016e6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01c6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b6016e6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52624/0 (socket says 172.21.15.16:52624) 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.393+0000 7f01c67fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42968/0 (socket says 172.21.15.16:42968) 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01c6ffd640 1 -- 172.21.15.16:0/4263144941 learned_addr learned my addr 172.21.15.16:0/4263144941 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0160b90 con 0x55b6012961b0 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0162000 con 0x55b6012bf070 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01c0161260 con 0x55b60127d9f0 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01d3ba5740 10 client.0 _ll_register_callbacks cb 0x7f01bc00a8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:49.396 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01c77fe640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601679cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.394+0000 7f01c67fc640 1 -- 172.21.15.16:0/4263144941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 msgr2=0x55b6016e6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.395+0000 7f01c67fc640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b6016e6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.395+0000 7f01c67fc640 1 -- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 msgr2=0x55b601679cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.395+0000 7f01c67fc640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601679cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.397 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.395+0000 7f01c67fc640 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b60167a340 con 0x55b6012961b0 2024-09-21T03:04:49.398 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.395+0000 7f01c67fc640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55b601667490 tx=0x7f01b80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3988f31635fb8652 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:49.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.396+0000 7f01aaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:04:49.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.396+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f01b80adee0 con 0x55b6012961b0 2024-09-21T03:04:49.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.396+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01c8095040 con 0x55b6012961b0 2024-09-21T03:04:49.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.396+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01b80b7a90 con 0x55b6012961b0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.397+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f01b80ae360 con 0x55b6012961b0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.397+0000 7f01d3ba5740 10 client.15915 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.397+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55b6016ee3a0 con 0x55b6012961b0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.398+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01c8095040 con 0x55b6012961b0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f01b8024040 con 0x55b6012961b0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 10 client.15915 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 10 client.15915 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 20 client.15915 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b6016eead0 0x55b6016f0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 4 client.15915 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.399+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b6015d7cd0 con 0x55b6016eead0 2024-09-21T03:04:49.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.400+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b6016eead0 0x55b6016f0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:49.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.401+0000 7f01c6ffd640 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b6016eead0 0x55b6016f0ef0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b20b53fec3448888 server_cookie=62c81395efdcd748 in_seq=0 out_seq=0 2024-09-21T03:04:49.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.402+0000 7f01aaff5640 10 client.15915 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:49.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.403+0000 7f01aaff5640 1 -- 172.21.15.16:0/4263144941 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b6015d7cd0 con 0x55b6016eead0 2024-09-21T03:04:49.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.403+0000 7f01aaff5640 10 client.15915 handle_command_reply: tid=0 2024-09-21T03:04:49.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.403+0000 7f01d3ba5740 1 client.15915 shutdown 2024-09-21T03:04:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.407+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 msgr2=0x55b6016e7120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.407+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x55b601667490 tx=0x7f01b80046c0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.407+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 shutdown_connections 2024-09-21T03:04:49.410 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.407+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55b6016eead0 0x55b6016f0ef0 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.407+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55b6012961b0 0x55b6016e7120 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.408+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55b6012bf070 0x55b601679cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.408+0000 7f01d3ba5740 1 --2- 172.21.15.16:0/4263144941 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b60127d9f0 0x55b6016e6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.408+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 >> 172.21.15.16:0/4263144941 conn(0x55b6012b3460 msgr2=0x55b601672e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.408+0000 7f01aaff5640 0 client.15915 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.409+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 shutdown_connections 2024-09-21T03:04:49.411 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.409+0000 7f01d3ba5740 1 -- 172.21.15.16:0/4263144941 wait complete. 2024-09-21T03:04:49.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.409+0000 7f01d3ba5740 20 client.15915 trim_cache size 0 max 16384 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 msgr2=0x7f01b00b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 0x7f01b00b8d90 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f01bc005fd0 tx=0x7f01bc005f60 comp rx=0 tx=0).stop 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 msgr2=0x7f01c00b6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f01b8006fd0 tx=0x7f01b800b8e0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 shutdown_connections 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01b00b6880 0x7f01b00b8d90 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01c00ab630 0x7f01c00b6dd0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01c00bbfd0 0x7f01c00bc3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 --2- 172.21.15.16:0/575780857 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01c00aace0 0x7f01c00b6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.412+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 >> 172.21.15.16:0/575780857 conn(0x7f01c001d0d0 msgr2=0x7f01c00a8af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.413+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 shutdown_connections 2024-09-21T03:04:49.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:49.413+0000 7f01adffb640 1 -- 172.21.15.16:0/575780857 wait complete. 2024-09-21T03:04:50.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:50 smithi123 ceph-mon[34645]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 465 MiB/s wr, 893 op/s 2024-09-21T03:04:50.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:50 smithi016 ceph-mon[31253]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 465 MiB/s wr, 893 op/s 2024-09-21T03:04:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:50 smithi018 ceph-mon[34587]: pgmap v455: 97 pgs: 97 active+clean; 40 GiB data, 122 GiB used, 951 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 465 MiB/s wr, 893 op/s 2024-09-21T03:04:51.429 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:04:51.430 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:04:51.869 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 -- 172.21.15.16:0/1975065013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838108d00 msgr2=0x7f483810f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 --2- 172.21.15.16:0/1975065013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838108d00 0x7f483810f530 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f4830009900 tx=0x7f483002a410 comp rx=0 tx=0).stop 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 -- 172.21.15.16:0/1975065013 shutdown_connections 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 --2- 172.21.15.16:0/1975065013 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838108d00 0x7f483810f530 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 --2- 172.21.15.16:0/1975065013 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48381083b0 0x7f48381087c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 --2- 172.21.15.16:0/1975065013 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4838102ac0 0x7f4838102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.498 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 -- 172.21.15.16:0/1975065013 >> 172.21.15.16:0/1975065013 conn(0x7f4838079790 msgr2=0x7f4838079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 -- 172.21.15.16:0/1975065013 shutdown_connections 2024-09-21T03:04:52.499 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.495+0000 7f483cbd6640 1 -- 172.21.15.16:0/1975065013 wait complete. 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.496+0000 7f483cbd6640 1 Processor -- start 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.496+0000 7f483cbd6640 1 -- start start 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48381083b0 0x7f483819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 0x7f483819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48381129c0 con 0x7f4838102ac0 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4838112840 con 0x7f48381083b0 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483cbd6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4838112b40 con 0x7f4838108d00 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 0x7f483819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42972/0 (socket says 172.21.15.16:42972) 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483659c640 1 -- 172.21.15.16:0/4279487907 learned_addr learned my addr 172.21.15.16:0/4279487907 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:52.500 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f482ffff640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48381083b0 0x7f483819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 -- 172.21.15.16:0/4279487907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48381083b0 msgr2=0x7f483819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48381083b0 0x7f483819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 -- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 msgr2=0x7f483819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f4836d9d640 1 -- 172.21.15.16:0/4279487907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f48381a0450 con 0x7f4838108d00 2024-09-21T03:04:52.501 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.497+0000 7f483659c640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:52.501 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:52 smithi016 ceph-mon[31253]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 603 MiB/s wr, 1.08k op/s 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f4836d9d640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 0x7f483819fd20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4830005940 tx=0x7f4830031a10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f482f7fe640 1 -- 172.21.15.16:0/4279487907 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f483003f030 con 0x7f4838108d00 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f482f7fe640 1 -- 172.21.15.16:0/4279487907 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f48300043d0 con 0x7f4838108d00 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f482f7fe640 1 -- 172.21.15.16:0/4279487907 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4830035070 con 0x7f4838108d00 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48381a06e0 con 0x7f4838108d00 2024-09-21T03:04:52.502 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.498+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48380ff530 con 0x7f4838108d00 2024-09-21T03:04:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.502+0000 7f482f7fe640 1 -- 172.21.15.16:0/4279487907 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4830042030 con 0x7f4838108d00 2024-09-21T03:04:52.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.502+0000 7f484314a740 1 Processor -- start 2024-09-21T03:04:52.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.502+0000 7f484314a740 1 -- start start 2024-09-21T03:04:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd63430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd63970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd63eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55584bcd9e60 con 0x55584b9bc350 2024-09-21T03:04:52.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55584bcc9290 con 0x55584b993490 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f484314a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584bccd0a0 con 0x55584b99b670 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482f7fe640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd63430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd63430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:42980/0 (socket says 172.21.15.16:42980) 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482ffff640 1 -- 172.21.15.16:0/2634946561 learned_addr learned my addr 172.21.15.16:0/2634946561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd63970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482f7fe640 1 -- 172.21.15.16:0/4279487907 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4830107990 con 0x7f4838108d00 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f4836d9d640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f482ffff640 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd63eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 -- 172.21.15.16:0/2634946561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 msgr2=0x55584bd63eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd63eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 -- 172.21.15.16:0/2634946561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 msgr2=0x55584bd63430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd63430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.503+0000 7f483659c640 1 -- 172.21.15.16:0/2634946561 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584ba27930 con 0x55584b993490 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.504+0000 7f4836d9d640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 0x7f48240b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4830006010 tx=0x7f483002a410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.504+0000 7f483659c640 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd63970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f482800e7d0 tx=0x7f482800ec90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ed07e81f901686d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.504+0000 7f482cff9640 1 -- 172.21.15.16:0/2634946561 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004d20 con 0x55584b993490 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.504+0000 7f482cff9640 1 -- 172.21.15.16:0/2634946561 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828005020 con 0x55584b993490 2024-09-21T03:04:52.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.504+0000 7f482cff9640 1 -- 172.21.15.16:0/2634946561 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482801b620 con 0x55584b993490 2024-09-21T03:04:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 -- 172.21.15.16:0/2634946561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 msgr2=0x55584bd63970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd63970 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f482800e7d0 tx=0x7f482800ec90 comp rx=0 tx=0).stop 2024-09-21T03:04:52.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 -- 172.21.15.16:0/2634946561 shutdown_connections 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd63eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd63970 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 --2- 172.21.15.16:0/2634946561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd63430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 -- 172.21.15.16:0/2634946561 >> 172.21.15.16:0/2634946561 conn(0x55584b9b0740 msgr2=0x55584bd61320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 -- 172.21.15.16:0/2634946561 shutdown_connections 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.505+0000 7f484314a740 1 -- 172.21.15.16:0/2634946561 wait complete. 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 -- start start 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 Processor -- start 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd6f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd7af00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55584bb2a890 con 0x55584b9bc350 2024-09-21T03:04:52.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55584bc63d40 con 0x55584b993490 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584bd71c10 con 0x55584b99b670 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f484314a740 10 client.0 _ll_register_callbacks cb 0x55584bcc7c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f482ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd6f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f483659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f483659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52672/0 (socket says 172.21.15.16:52672) 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f483659c640 1 -- 172.21.15.16:0/3559385086 learned_addr learned my addr 172.21.15.16:0/3559385086 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.506+0000 7f482ffff640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd7af00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 -- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 msgr2=0x55584bd6f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd6f090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 -- 172.21.15.16:0/3559385086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 msgr2=0x55584bd7af00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd7af00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 -- 172.21.15.16:0/3559385086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584bd7b510 con 0x55584b99b670 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f483659c640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4828005620 tx=0x7f4828005650 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d31b331e7108cb59 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:04:52.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4828004c50 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4828004df0 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.507+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4828005020 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4828011a20 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 10 client.25570 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55584bd7de40 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4828005020 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4828004250 con 0x55584b99b670 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 10 client.25570 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 10 client.25570 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 20 client.25570 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b97acd0 0x55584bd81250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 4 client.25570 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55584bcdbdb0 con 0x55584b97acd0 2024-09-21T03:04:52.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.508+0000 7f482ffff640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b97acd0 0x55584bd81250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.509+0000 7f481dffb640 10 client.25570 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:52.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.509+0000 7f482ffff640 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b97acd0 0x55584bd81250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=accd5f568b2990f2 server_cookie=350e89f946cf4858 in_seq=0 out_seq=0 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.509+0000 7f481dffb640 1 -- 172.21.15.16:0/3559385086 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55584bcdbdb0 con 0x55584b97acd0 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.509+0000 7f481dffb640 10 client.25570 handle_command_reply: tid=0 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 client.25570 shutdown 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 msgr2=0x55584bd6f610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f4828005620 tx=0x7f4828005650 comp rx=0 tx=0).stop 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 shutdown_connections 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b97acd0 0x55584bd81250 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b9bc350 0x55584bd7af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b99b670 0x55584bd6f610 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f481dffb640 0 client.25570 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 --2- 172.21.15.16:0/3559385086 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b993490 0x55584bd6f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 >> 172.21.15.16:0/3559385086 conn(0x55584b9b0740 msgr2=0x55584bd66610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 shutdown_connections 2024-09-21T03:04:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 1 -- 172.21.15.16:0/3559385086 wait complete. 2024-09-21T03:04:52.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.510+0000 7f484314a740 20 client.25570 trim_cache size 0 max 16384 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.529+0000 7f484314a740 1 Processor -- start 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.529+0000 7f484314a740 1 -- start start 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.529+0000 7f484314a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.529+0000 7f484314a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd7ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.529+0000 7f484314a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd68510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f484314a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ab550 con 0x55584b99b670 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f484314a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48381addd0 con 0x55584b9bc350 2024-09-21T03:04:52.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f484314a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48381abbc0 con 0x55584b993490 2024-09-21T03:04:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52676/0 (socket says 172.21.15.16:52676) 2024-09-21T03:04:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 -- 172.21.15.16:0/2197364134 learned_addr learned my addr 172.21.15.16:0/2197364134 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:52.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f483659c640 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd7ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd68510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 -- 172.21.15.16:0/2197364134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 msgr2=0x55584bd68510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd68510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 -- 172.21.15.16:0/2197364134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 msgr2=0x55584bd7ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd7ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 -- 172.21.15.16:0/2197364134 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584bd7b340 con 0x55584b993490 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.530+0000 7f482ffff640 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4820005fd0 tx=0x7f4820013040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d7cbbb25a1d062f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.531+0000 7f481ffff640 1 -- 172.21.15.16:0/2197364134 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820005220 con 0x55584b993490 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.531+0000 7f481ffff640 1 -- 172.21.15.16:0/2197364134 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820031070 con 0x55584b993490 2024-09-21T03:04:52.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.531+0000 7f481ffff640 1 -- 172.21.15.16:0/2197364134 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4820010430 con 0x55584b993490 2024-09-21T03:04:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.531+0000 7f484314a740 1 -- 172.21.15.16:0/2197364134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 msgr2=0x55584bd66c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.531+0000 7f484314a740 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f4820005fd0 tx=0x7f4820013040 comp rx=0 tx=0).stop 2024-09-21T03:04:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 -- 172.21.15.16:0/2197364134 shutdown_connections 2024-09-21T03:04:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd7ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd68510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 --2- 172.21.15.16:0/2197364134 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd66c40 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 -- 172.21.15.16:0/2197364134 >> 172.21.15.16:0/2197364134 conn(0x55584b9b0740 msgr2=0x55584bd66530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 -- 172.21.15.16:0/2197364134 shutdown_connections 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 -- 172.21.15.16:0/2197364134 wait complete. 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 -- start start 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.532+0000 7f484314a740 1 Processor -- start 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ac290 con 0x55584b99b670 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ad700 con 0x55584b9bc350 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f48381ac960 con 0x55584b993490 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f484314a740 10 client.0 _ll_register_callbacks cb 0x7f4820009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:52.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f483659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:52682/0 (socket says 172.21.15.16:52682) 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f483659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43016/0 (socket says 172.21.15.16:43016) 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 -- 172.21.15.16:0/4117259244 learned_addr learned my addr 172.21.15.16:0/4117259244 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 -- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 msgr2=0x55584bd755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 -- 172.21.15.16:0/4117259244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 msgr2=0x55584bd750a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 -- 172.21.15.16:0/4117259244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584bd75d20 con 0x55584b993490 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f483659c640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f482ffff640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4820009a60 tx=0x7f4820009910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5654266404372fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.533+0000 7f481dffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f482003a070 con 0x55584b993490 2024-09-21T03:04:52.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f4820005200 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4820031070 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f482004c050 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 10 client.25576 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55584bdec9e0 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4820031070 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f4820004b90 con 0x55584b993490 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 10 client.25576 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 10 client.25576 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 20 client.25576 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b950080 0x55584bdeefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 4 client.25576 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55584bcd7860 con 0x55584b950080 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.534+0000 7f483659c640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b950080 0x55584bdeefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.535+0000 7f481dffb640 10 client.25576 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:52.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.535+0000 7f483659c640 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b950080 0x55584bdeefb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52c9fd1bcae9d99d server_cookie=382fcd3fb3d9d16f in_seq=0 out_seq=0 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f481dffb640 1 -- 172.21.15.16:0/4117259244 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55584bcd7860 con 0x55584b950080 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f481dffb640 10 client.25576 handle_command_reply: tid=0 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 client.25576 shutdown 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 msgr2=0x55584bd77630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f4820009a60 tx=0x7f4820009910 comp rx=0 tx=0).stop 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 shutdown_connections 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584b950080 0x55584bdeefb0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584b99b670 0x55584bd750a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584b9bc350 0x55584bd755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 --2- 172.21.15.16:0/4117259244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584b993490 0x55584bd77630 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 >> 172.21.15.16:0/4117259244 conn(0x55584b9b0740 msgr2=0x55584bd60c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.537+0000 7f481dffb640 0 client.25576 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:52.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 shutdown_connections 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f484314a740 1 -- 172.21.15.16:0/4117259244 wait complete. 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f484314a740 20 client.25576 trim_cache size 0 max 16384 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "7357d59f-9636-4aa8-bd7f-5c495426dcb6", 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:04:52.543 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 msgr2=0x7f48240b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 0x7f48240b8b90 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f4830006010 tx=0x7f483002a410 comp rx=0 tx=0).stop 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 msgr2=0x7f483819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 0x7f483819fd20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f4830005940 tx=0x7f4830031a10 comp rx=0 tx=0).stop 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 shutdown_connections 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f48240b6680 0x7f48240b8b90 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4838108d00 0x7f483819fd20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f48381083b0 0x7f483819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 --2- 172.21.15.16:0/4279487907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4838102ac0 0x7f483819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 >> 172.21.15.16:0/4279487907 conn(0x7f4838079790 msgr2=0x7f4838101a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.538+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 shutdown_connections 2024-09-21T03:04:52.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:52.539+0000 7f483cbd6640 1 -- 172.21.15.16:0/4279487907 wait complete. 2024-09-21T03:04:52.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:52 smithi018 ceph-mon[34587]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 603 MiB/s wr, 1.08k op/s 2024-09-21T03:04:52.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:52 smithi123 ceph-mon[34645]: pgmap v456: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 603 MiB/s wr, 1.08k op/s 2024-09-21T03:04:53.448 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:04:53.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:53 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:04:53.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:53 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:53.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:53 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:04:53.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:53 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:04:53.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:53 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:53.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:53 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:04:53.938 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:04:53.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:53 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:04:53.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:53 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:53.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:53 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:04:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 -- 172.21.15.16:0/363743129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 msgr2=0x7f0198104dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 --2- 172.21.15.16:0/363743129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f0198104dc0 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f018c009900 tx=0x7f018c02a410 comp rx=0 tx=0).stop 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 -- 172.21.15.16:0/363743129 shutdown_connections 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 --2- 172.21.15.16:0/363743129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f019810de10 0x7f0198110300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 --2- 172.21.15.16:0/363743129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0198105300 0x7f019810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 --2- 172.21.15.16:0/363743129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f0198104dc0 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 -- 172.21.15.16:0/363743129 >> 172.21.15.16:0/363743129 conn(0x7f0198079730 msgr2=0x7f0198079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 -- 172.21.15.16:0/363743129 shutdown_connections 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.655+0000 7f019ec85640 1 -- 172.21.15.16:0/363743129 wait complete. 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 Processor -- start 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 -- start start 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f01981026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 0x7f0198102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0198113ae0 con 0x7f01981049b0 2024-09-21T03:04:54.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0198113960 con 0x7f0198105300 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ec85640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0198113c60 con 0x7f019810de10 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019d222640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019d222640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:38298/0 (socket says 172.21.15.16:38298) 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019d222640 1 -- 172.21.15.16:0/1349095004 learned_addr learned my addr 172.21.15.16:0/1349095004 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f0197fff640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 0x7f0198102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.656+0000 7f019ca21640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f01981026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 -- 172.21.15.16:0/1349095004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 msgr2=0x7f0198102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 0x7f0198102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 -- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 msgr2=0x7f01981026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f01981026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 -- 172.21.15.16:0/1349095004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f01981a8770 con 0x7f019810de10 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019ca21640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f01981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f0197fff640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 0x7f0198102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:54.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019d222640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f019000eb10 tx=0x7f019000c2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f0195ffb640 1 -- 172.21.15.16:0/1349095004 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190004b20 con 0x7f019810de10 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f01981a8a60 con 0x7f019810de10 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f0195ffb640 1 -- 172.21.15.16:0/1349095004 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0190004cc0 con 0x7f019810de10 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.657+0000 7f0195ffb640 1 -- 172.21.15.16:0/1349095004 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0190010670 con 0x7f019810de10 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.658+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01981a8f40 con 0x7f019810de10 2024-09-21T03:04:54.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.660+0000 7f0195ffb640 1 -- 172.21.15.16:0/1349095004 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f01900108f0 con 0x7f019810de10 2024-09-21T03:04:54.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 Processor -- start 2024-09-21T03:04:54.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 -- start start 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 0x56167e674270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e6747b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e674cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56167e5eae50 con 0x56167e2ac390 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56167e5da280 con 0x56167e2a41b0 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f01a51f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56167e5de090 con 0x56167e2cd070 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f0195ffb640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 0x7f01880b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f0197fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e674cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f019ca21640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e6747b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f0197fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e674cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49234/0 (socket says 172.21.15.16:49234) 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f0197fff640 1 -- 172.21.15.16:0/1171875463 learned_addr learned my addr 172.21.15.16:0/1171875463 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:54.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.661+0000 7f019d222640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 0x7f01880b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0197fff640 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 0x56167e674270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 -- 172.21.15.16:0/1171875463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 msgr2=0x56167e674270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 0x56167e674270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 -- 172.21.15.16:0/1171875463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 msgr2=0x56167e674cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e674cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 -- 172.21.15.16:0/1171875463 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167e338650 con 0x56167e2a41b0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0197fff640 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 0x56167e674270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0195ffb640 1 -- 172.21.15.16:0/1349095004 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f01900e7d40 con 0x7f019810de10 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019ca21640 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e6747b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a930 tx=0x7f018c004180 comp rx=0 tx=0).ready entity=mon.2 client_cookie=17ec049330912754 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f019d222640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 0x7f01880b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f01900ad8b0 tx=0x7f0190013000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0182ffd640 1 -- 172.21.15.16:0/1171875463 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035030 con 0x56167e2a41b0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0182ffd640 1 -- 172.21.15.16:0/1171875463 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018c0054f0 con 0x56167e2a41b0 2024-09-21T03:04:54.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.662+0000 7f0182ffd640 1 -- 172.21.15.16:0/1171875463 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c0035f0 con 0x56167e2a41b0 2024-09-21T03:04:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 -- 172.21.15.16:0/1171875463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 msgr2=0x56167e6747b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e6747b0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f018c02a930 tx=0x7f018c004180 comp rx=0 tx=0).stop 2024-09-21T03:04:54.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 -- 172.21.15.16:0/1171875463 shutdown_connections 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e674cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e6747b0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 --2- 172.21.15.16:0/1171875463 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2cd070 0x56167e674270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 -- 172.21.15.16:0/1171875463 >> 172.21.15.16:0/1171875463 conn(0x56167e2c1460 msgr2=0x56167e672160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 -- 172.21.15.16:0/1171875463 shutdown_connections 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.663+0000 7f01a51f9740 1 -- 172.21.15.16:0/1171875463 wait complete. 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.664+0000 7f01a51f9740 1 -- start start 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.664+0000 7f01a51f9740 1 Processor -- start 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e67feb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 0x56167e68bce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56167e43b700 con 0x56167e2ac390 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56167e575110 con 0x56167e2cd070 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56167e6829f0 con 0x56167e2a41b0 2024-09-21T03:04:54.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f01a51f9740 10 client.0 _ll_register_callbacks cb 0x56167e58e990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f019ca21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e67feb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67f970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:38310/0 (socket says 172.21.15.16:38310) 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f019ca21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e67feb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49250/0 (socket says 172.21.15.16:49250) 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 -- 172.21.15.16:0/2703546215 learned_addr learned my addr 172.21.15.16:0/2703546215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 0x56167e68bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 -- 172.21.15.16:0/2703546215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 msgr2=0x56167e67f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 -- 172.21.15.16:0/2703546215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 msgr2=0x56167e67feb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e67feb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.665+0000 7f0197fff640 1 -- 172.21.15.16:0/2703546215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167e5a2c40 con 0x56167e2cd070 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0197fff640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 0x56167e68bce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x56167e6821c0 tx=0x7f018400bd60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8775c37beb154c56 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0177fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018400e560 con 0x56167e2cd070 2024-09-21T03:04:54.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f018400e700 con 0x56167e2cd070 2024-09-21T03:04:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f018c0054f0 con 0x56167e2cd070 2024-09-21T03:04:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f01840129e0 con 0x56167e2cd070 2024-09-21T03:04:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f01a51f9740 10 client.25622 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:54.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56167e68e780 con 0x56167e2cd070 2024-09-21T03:04:54.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.666+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f018c0054f0 con 0x56167e2cd070 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f0184028070 con 0x56167e2cd070 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 10 client.25622 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 10 client.25622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 20 client.25622 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e28b9f0 0x56167e691cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 4 client.25622 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56167e5ecda0 con 0x56167e28b9f0 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f019ca21640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e28b9f0 0x56167e691cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f0177fff640 10 client.25622 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.667+0000 7f019ca21640 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e28b9f0 0x56167e691cf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bcfe848e5f6a0bbe server_cookie=8258223cc7012c3b in_seq=0 out_seq=0 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.668+0000 7f0177fff640 1 -- 172.21.15.16:0/2703546215 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56167e5ecda0 con 0x56167e28b9f0 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.668+0000 7f0177fff640 10 client.25622 handle_command_reply: tid=0 2024-09-21T03:04:54.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.668+0000 7f01a51f9740 1 client.25622 shutdown 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 msgr2=0x56167e68bce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 0x56167e68bce0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x56167e6821c0 tx=0x7f018400bd60 comp rx=0 tx=0).stop 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 shutdown_connections 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e28b9f0 0x56167e691cf0 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2ac390 0x56167e67feb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2cd070 0x56167e68bce0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f0177fff640 0 client.25622 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 --2- 172.21.15.16:0/2703546215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 >> 172.21.15.16:0/2703546215 conn(0x56167e2c1460 msgr2=0x56167e677450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 shutdown_connections 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 1 -- 172.21.15.16:0/2703546215 wait complete. 2024-09-21T03:04:54.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.669+0000 7f01a51f9740 20 client.25622 trim_cache size 0 max 16384 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 Processor -- start 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 -- start start 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2c1460 0x56167e672190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67cb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01981af770 con 0x56167e28b9f0 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01981b1ff0 con 0x56167e2c1460 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.688+0000 7f01a51f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01981afde0 con 0x56167e2a41b0 2024-09-21T03:04:54.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f019ca21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67cb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f019ca21640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49266/0 (socket says 172.21.15.16:49266) 2024-09-21T03:04:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f019ca21640 1 -- 172.21.15.16:0/3312663254 learned_addr learned my addr 172.21.15.16:0/3312663254 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2c1460 0x56167e672190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 -- 172.21.15.16:0/3312663254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2c1460 msgr2=0x56167e672190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2c1460 0x56167e672190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 -- 172.21.15.16:0/3312663254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 msgr2=0x56167e68bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 -- 172.21.15.16:0/3312663254 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56167e676880 con 0x56167e2a41b0 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f019ca21640 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f0197fff640 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f018400b520 tx=0x7f018400b390 comp rx=0 tx=0).ready entity=mon.1 client_cookie=95c18dc955837de7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.689+0000 7f01827fc640 1 -- 172.21.15.16:0/3312663254 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018402e040 con 0x56167e2a41b0 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.690+0000 7f01827fc640 1 -- 172.21.15.16:0/3312663254 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0184021b70 con 0x56167e2a41b0 2024-09-21T03:04:54.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.690+0000 7f01827fc640 1 -- 172.21.15.16:0/3312663254 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018400ebb0 con 0x56167e2a41b0 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- 172.21.15.16:0/3312663254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 msgr2=0x56167e67cb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67cb60 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f018400b520 tx=0x7f018400b390 comp rx=0 tx=0).stop 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- 172.21.15.16:0/3312663254 shutdown_connections 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e28b9f0 0x56167e68bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e2a41b0 0x56167e67cb60 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 --2- 172.21.15.16:0/3312663254 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2c1460 0x56167e672190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- 172.21.15.16:0/3312663254 >> 172.21.15.16:0/3312663254 conn(0x56167e680030 msgr2=0x56167e671a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- 172.21.15.16:0/3312663254 shutdown_connections 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- 172.21.15.16:0/3312663254 wait complete. 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 -- start start 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.691+0000 7f01a51f9740 1 Processor -- start 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e28b9f0 0x56167e683cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2c1460 0x56167e684770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f01981b04b0 con 0x56167e2c1460 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01981b1920 con 0x56167e2a41b0 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f01981b0b80 con 0x56167e28b9f0 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f01a51f9740 10 client.0 _ll_register_callbacks cb 0x7f0184012810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f019ca21640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f019ca21640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:60360/0 (socket says 172.21.15.16:60360) 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f0197fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2c1460 0x56167e684770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f019ca21640 1 -- 172.21.15.16:0/60639275 learned_addr learned my addr 172.21.15.16:0/60639275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:04:54.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.692+0000 7f0197fff640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e28b9f0 0x56167e683cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 -- 172.21.15.16:0/60639275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e28b9f0 msgr2=0x56167e683cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e28b9f0 0x56167e683cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 -- 172.21.15.16:0/60639275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2c1460 msgr2=0x56167e684770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2c1460 0x56167e684770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 -- 172.21.15.16:0/60639275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56167e6fb7c0 con 0x56167e2a41b0 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f019ca21640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f018c006010 tx=0x7f018c005a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f6529e117ad4403 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:04:54.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f0177fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f018c035040 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f018c003b40 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0184021b70 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f018c062050 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f01a51f9740 10 client.25628 fetch_fsmap learned FSMap version 20 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.693+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56167e6fcae0 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0184021b70 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f018c03b430 con 0x56167e2a41b0 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 10 client.25628 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:04:54.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 10 client.25628 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 20 client.25628 populate_metadata read hostname 'smithi016' 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e2ac390 0x56167e6feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 4 client.25628 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56167e5e8850 con 0x56167e2ac390 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f0197fff640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e2ac390 0x56167e6feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.694+0000 7f0177fff640 10 client.25628 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f0197fff640 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e2ac390 0x56167e6feef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18024fab9f70245a server_cookie=91f52bf0a65d2647 in_seq=0 out_seq=0 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f0177fff640 1 -- 172.21.15.16:0/60639275 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2842 (crc 0 0 0) 0x56167e5e8850 con 0x56167e2ac390 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f0177fff640 10 client.25628 handle_command_reply: tid=0 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 client.25628 shutdown 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 msgr2=0x56167e684230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f018c006010 tx=0x7f018c005a40 comp rx=0 tx=0).stop 2024-09-21T03:04:54.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 shutdown_connections 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56167e2ac390 0x56167e6feef0 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56167e2c1460 0x56167e684770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56167e2a41b0 0x56167e684230 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f0177fff640 0 client.25628 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 --2- 172.21.15.16:0/60639275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56167e28b9f0 0x56167e683cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 >> 172.21.15.16:0/60639275 conn(0x56167e680030 msgr2=0x56167e6751d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 shutdown_connections 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 1 -- 172.21.15.16:0/60639275 wait complete. 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.695+0000 7f01a51f9740 20 client.25628 trim_cache size 0 max 16384 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:04:54.701 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "7357d59f-9636-4aa8-bd7f-5c495426dcb6", 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.705 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:04:54.706 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 msgr2=0x7f01880b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 0x7f01880b8c10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f01900ad8b0 tx=0x7f0190013000 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 msgr2=0x7f0198103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f019000eb10 tx=0x7f019000c2d0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 shutdown_connections 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f01880b6700 0x7f01880b8c10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f019810de10 0x7f0198103170 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0198105300 0x7f0198102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 --2- 172.21.15.16:0/1349095004 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f01981049b0 0x7f01981026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.696+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 >> 172.21.15.16:0/1349095004 conn(0x7f0198079730 msgr2=0x7f01981007b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.697+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 shutdown_connections 2024-09-21T03:04:54.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:04:54.697+0000 7f019ec85640 1 -- 172.21.15.16:0/1349095004 wait complete. 2024-09-21T03:04:54.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:54 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:54.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:54 smithi016 ceph-mon[31253]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 503 MiB/s wr, 669 op/s 2024-09-21T03:04:54.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:54 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:54.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:54 smithi018 ceph-mon[34587]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 503 MiB/s wr, 669 op/s 2024-09-21T03:04:54.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:54 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:04:54.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:54 smithi123 ceph-mon[34645]: pgmap v457: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 503 MiB/s wr, 669 op/s 2024-09-21T03:04:56.211 INFO:tasks.cephfs.filesystem:scrub status for tag:7357d59f-9636-4aa8-bd7f-5c495426dcb6 - {'path': '/', 'tag': '7357d59f-9636-4aa8-bd7f-5c495426dcb6', 'options': 'recursive,force'} 2024-09-21T03:04:56.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:56 smithi016 ceph-mon[31253]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 568 MiB/s wr, 817 op/s 2024-09-21T03:04:56.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:56 smithi018 ceph-mon[34587]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 568 MiB/s wr, 817 op/s 2024-09-21T03:04:56.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:56 smithi123 ceph-mon[34645]: pgmap v458: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 568 MiB/s wr, 817 op/s 2024-09-21T03:04:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:58 smithi018 ceph-mon[34587]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 419 MiB/s wr, 725 op/s 2024-09-21T03:04:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:04:58 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:04:58.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:58 smithi123 ceph-mon[34645]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 419 MiB/s wr, 725 op/s 2024-09-21T03:04:58.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:04:58 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:04:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:58 smithi016 ceph-mon[31253]: pgmap v459: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 419 MiB/s wr, 725 op/s 2024-09-21T03:04:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:04:58 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:05:00.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:00 smithi016 ceph-mon[31253]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 372 MiB/s wr, 686 op/s 2024-09-21T03:05:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:00 smithi018 ceph-mon[34587]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 372 MiB/s wr, 686 op/s 2024-09-21T03:05:00.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:00 smithi123 ceph-mon[34645]: pgmap v460: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 372 MiB/s wr, 686 op/s 2024-09-21T03:05:02.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:02 smithi018 ceph-mon[34587]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 431 MiB/s wr, 719 op/s 2024-09-21T03:05:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:02 smithi123 ceph-mon[34645]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 431 MiB/s wr, 719 op/s 2024-09-21T03:05:03.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:02 smithi016 ceph-mon[31253]: pgmap v461: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 431 MiB/s wr, 719 op/s 2024-09-21T03:05:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:04 smithi018 ceph-mon[34587]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 285 MiB/s wr, 417 op/s 2024-09-21T03:05:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:04 smithi123 ceph-mon[34645]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 285 MiB/s wr, 417 op/s 2024-09-21T03:05:05.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:04 smithi016 ceph-mon[31253]: pgmap v462: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 285 MiB/s wr, 417 op/s 2024-09-21T03:05:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:06 smithi018 ceph-mon[34587]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 401 MiB/s wr, 420 op/s 2024-09-21T03:05:06.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:06 smithi123 ceph-mon[34645]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 401 MiB/s wr, 420 op/s 2024-09-21T03:05:07.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:06 smithi016 ceph-mon[31253]: pgmap v463: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 401 MiB/s wr, 420 op/s 2024-09-21T03:05:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:08 smithi018 ceph-mon[34587]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 956 KiB/s rd, 374 MiB/s wr, 261 op/s 2024-09-21T03:05:08.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:08 smithi123 ceph-mon[34645]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 956 KiB/s rd, 374 MiB/s wr, 261 op/s 2024-09-21T03:05:09.020 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:08 smithi016 ceph-mon[31253]: pgmap v464: 97 pgs: 97 active+clean; 40 GiB data, 123 GiB used, 950 GiB / 1.0 TiB avail; 956 KiB/s rd, 374 MiB/s wr, 261 op/s 2024-09-21T03:05:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:10 smithi018 ceph-mon[34587]: pgmap v465: 97 pgs: 97 active+clean; 41 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 956 KiB/s rd, 390 MiB/s wr, 390 op/s 2024-09-21T03:05:10.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:10 smithi123 ceph-mon[34645]: pgmap v465: 97 pgs: 97 active+clean; 41 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 956 KiB/s rd, 390 MiB/s wr, 390 op/s 2024-09-21T03:05:11.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:10 smithi016 ceph-mon[31253]: pgmap v465: 97 pgs: 97 active+clean; 41 GiB data, 124 GiB used, 949 GiB / 1.0 TiB avail; 956 KiB/s rd, 390 MiB/s wr, 390 op/s 2024-09-21T03:05:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:12 smithi123 ceph-mon[34645]: pgmap v466: 97 pgs: 97 active+clean; 43 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 577 MiB/s wr, 1.24k op/s 2024-09-21T03:05:13.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:12 smithi016 ceph-mon[31253]: pgmap v466: 97 pgs: 97 active+clean; 43 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 577 MiB/s wr, 1.24k op/s 2024-09-21T03:05:13.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:12 smithi018 ceph-mon[34587]: pgmap v466: 97 pgs: 97 active+clean; 43 GiB data, 129 GiB used, 944 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 577 MiB/s wr, 1.24k op/s 2024-09-21T03:05:15.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:14 smithi018 ceph-mon[34587]: pgmap v467: 97 pgs: 97 active+clean; 43 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1003 KiB/s rd, 504 MiB/s wr, 1.36k op/s 2024-09-21T03:05:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:14 smithi123 ceph-mon[34645]: pgmap v467: 97 pgs: 97 active+clean; 43 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1003 KiB/s rd, 504 MiB/s wr, 1.36k op/s 2024-09-21T03:05:15.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:14 smithi016 ceph-mon[31253]: pgmap v467: 97 pgs: 97 active+clean; 43 GiB data, 130 GiB used, 943 GiB / 1.0 TiB avail; 1003 KiB/s rd, 504 MiB/s wr, 1.36k op/s 2024-09-21T03:05:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:16 smithi018 ceph-mon[34587]: pgmap v468: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 MiB/s wr, 2.43k op/s 2024-09-21T03:05:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:16 smithi123 ceph-mon[34645]: pgmap v468: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 MiB/s wr, 2.43k op/s 2024-09-21T03:05:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:16 smithi016 ceph-mon[31253]: pgmap v468: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 618 MiB/s wr, 2.43k op/s 2024-09-21T03:05:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:18 smithi018 ceph-mon[34587]: pgmap v469: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1000 KiB/s rd, 539 MiB/s wr, 2.85k op/s 2024-09-21T03:05:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:18 smithi123 ceph-mon[34645]: pgmap v469: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1000 KiB/s rd, 539 MiB/s wr, 2.85k op/s 2024-09-21T03:05:19.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:18 smithi016 ceph-mon[31253]: pgmap v469: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1000 KiB/s rd, 539 MiB/s wr, 2.85k op/s 2024-09-21T03:05:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:20 smithi018 ceph-mon[34587]: pgmap v470: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 505 MiB/s wr, 3.02k op/s 2024-09-21T03:05:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:20 smithi123 ceph-mon[34645]: pgmap v470: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 505 MiB/s wr, 3.02k op/s 2024-09-21T03:05:21.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:20 smithi016 ceph-mon[31253]: pgmap v470: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 505 MiB/s wr, 3.02k op/s 2024-09-21T03:05:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:22 smithi018 ceph-mon[34587]: pgmap v471: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 525 MiB/s wr, 3.38k op/s 2024-09-21T03:05:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:22 smithi123 ceph-mon[34645]: pgmap v471: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 525 MiB/s wr, 3.38k op/s 2024-09-21T03:05:23.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:22 smithi016 ceph-mon[31253]: pgmap v471: 97 pgs: 97 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 525 MiB/s wr, 3.38k op/s 2024-09-21T03:05:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:24 smithi018 ceph-mon[34587]: pgmap v472: 97 pgs: 97 active+clean; 43 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 767 KiB/s rd, 330 MiB/s wr, 2.52k op/s 2024-09-21T03:05:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:24 smithi123 ceph-mon[34645]: pgmap v472: 97 pgs: 97 active+clean; 43 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 767 KiB/s rd, 330 MiB/s wr, 2.52k op/s 2024-09-21T03:05:25.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:24 smithi016 ceph-mon[31253]: pgmap v472: 97 pgs: 97 active+clean; 43 GiB data, 132 GiB used, 940 GiB / 1.0 TiB avail; 767 KiB/s rd, 330 MiB/s wr, 2.52k op/s 2024-09-21T03:05:26.213 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:05:26.603 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:05:27.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:26 smithi016 ceph-mon[31253]: pgmap v473: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 454 MiB/s wr, 2.41k op/s 2024-09-21T03:05:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:26 smithi018 ceph-mon[34587]: pgmap v473: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 454 MiB/s wr, 2.41k op/s 2024-09-21T03:05:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:26 smithi123 ceph-mon[34645]: pgmap v473: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 939 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 454 MiB/s wr, 2.41k op/s 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 -- 172.21.15.16:0/4161599811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 msgr2=0x7feec410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 --2- 172.21.15.16:0/4161599811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec410f520 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7feebc009ca0 tx=0x7feebc02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 -- 172.21.15.16:0/4161599811 shutdown_connections 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 --2- 172.21.15.16:0/4161599811 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec410f520 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 --2- 172.21.15.16:0/4161599811 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec41087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 --2- 172.21.15.16:0/4161599811 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 -- 172.21.15.16:0/4161599811 >> 172.21.15.16:0/4161599811 conn(0x7feec4079730 msgr2=0x7feec4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.323 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 -- 172.21.15.16:0/4161599811 shutdown_connections 2024-09-21T03:05:27.324 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.321+0000 7feecbd9f640 1 -- 172.21.15.16:0/4161599811 wait complete. 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 Processor -- start 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 -- start start 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec419aa90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec419fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113a80 con 0x7feec4108cf0 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113900 con 0x7feec4102a60 2024-09-21T03:05:27.325 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.322+0000 7feecbd9f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec4113c00 con 0x7feec41083a0 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feeca33c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec419fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec933a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec419aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:56986/0 (socket says 172.21.15.16:56986) 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec933a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec419aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42762/0 (socket says 172.21.15.16:42762) 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 -- 172.21.15.16:0/3754246222 learned_addr learned my addr 172.21.15.16:0/3754246222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 -- 172.21.15.16:0/3754246222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 msgr2=0x7feec419aa90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec419aa90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 -- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 msgr2=0x7feec419fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec419fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 -- 172.21.15.16:0/3754246222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feec41a0460 con 0x7feec4102a60 2024-09-21T03:05:27.326 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feec9b3b640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feec000ea60 tx=0x7feec000ef20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feebaffd640 1 -- 172.21.15.16:0/3754246222 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec000cde0 con 0x7feec4102a60 2024-09-21T03:05:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feebaffd640 1 -- 172.21.15.16:0/3754246222 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00035f0 con 0x7feec4102a60 2024-09-21T03:05:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.323+0000 7feebaffd640 1 -- 172.21.15.16:0/3754246222 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec0010690 con 0x7feec4102a60 2024-09-21T03:05:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.324+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feec41a0750 con 0x7feec4102a60 2024-09-21T03:05:27.327 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.324+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feec41a6eb0 con 0x7feec4102a60 2024-09-21T03:05:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 Processor -- start 2024-09-21T03:05:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feebaffd640 1 -- 172.21.15.16:0/3754246222 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7feec00037a0 con 0x7feec4102a60 2024-09-21T03:05:27.328 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 -- start start 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 0x558c92eba650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebb0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c92e30e50 con 0x558c92af2390 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558c92e20280 con 0x558c92b13070 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.327+0000 7feed2313740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c92e24090 con 0x558c92aea1b0 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec933a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebb0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec933a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebb0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48366/0 (socket says 172.21.15.16:48366) 2024-09-21T03:05:27.330 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42770/0 (socket says 172.21.15.16:42770) 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec933a640 1 -- 172.21.15.16:0/4125498354 learned_addr learned my addr 172.21.15.16:0/4125498354 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feeca33c640 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 0x558c92eba650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 -- 172.21.15.16:0/4125498354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 msgr2=0x558c92eba650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 0x558c92eba650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 -- 172.21.15.16:0/4125498354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 msgr2=0x558c92ebb0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebb0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feebaffd640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 0x7feeb00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 -- 172.21.15.16:0/4125498354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c92b7e650 con 0x558c92aea1b0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feeca33c640 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 0x558c92eba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feeca33c640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 0x7feeb00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feec9b3b640 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feec00e5420 tx=0x7feec00aa230 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b48c354c6fd05400 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.328+0000 7feebaffd640 1 -- 172.21.15.16:0/3754246222 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7feec00e7cb0 con 0x7feec4102a60 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.329+0000 7feeaffff640 1 -- 172.21.15.16:0/4125498354 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ad8f0 con 0x558c92aea1b0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.329+0000 7feeaffff640 1 -- 172.21.15.16:0/4125498354 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.331 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.329+0000 7feeaffff640 1 -- 172.21.15.16:0/4125498354 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ad290 con 0x558c92aea1b0 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- 172.21.15.16:0/4125498354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 msgr2=0x558c92ebab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feeca33c640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 0x7feeb00b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feebc006010 tx=0x7feebc009460 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7feec00e5420 tx=0x7feec00aa230 comp rx=0 tx=0).stop 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- 172.21.15.16:0/4125498354 shutdown_connections 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebb0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.334 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92ebab90 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 --2- 172.21.15.16:0/4125498354 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92b13070 0x558c92eba650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- 172.21.15.16:0/4125498354 >> 172.21.15.16:0/4125498354 conn(0x558c92b07460 msgr2=0x558c92eb8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- 172.21.15.16:0/4125498354 shutdown_connections 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- 172.21.15.16:0/4125498354 wait complete. 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 -- start start 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.330+0000 7feed2313740 1 Processor -- start 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92eca0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92ecab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558c92c81700 con 0x558c92b13070 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558c92dbb110 con 0x558c92af2390 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558c92ecd160 con 0x558c92aea1b0 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feed2313740 10 client.0 _ll_register_callbacks cb 0x558c92dd4990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:27.335 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec933a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feeca33c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92ecab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92eca0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feeca33c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92ecab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48382/0 (socket says 172.21.15.16:48382) 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec933a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:56998/0 (socket says 172.21.15.16:56998) 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feeca33c640 1 -- 172.21.15.16:0/2077934032 learned_addr learned my addr 172.21.15.16:0/2077934032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 -- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 msgr2=0x558c92eca5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 -- 172.21.15.16:0/2077934032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 msgr2=0x558c92ecab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92ecab30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec9b3b640 1 -- 172.21.15.16:0/2077934032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c92de8c40 con 0x558c92aea1b0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.331+0000 7feec933a640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feec9b3b640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92eca0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feec00d1000 tx=0x7feec00e7ff0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a85b8ab40272edea server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ad470 con 0x558c92aea1b0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7feec00a9470 con 0x558c92aea1b0 2024-09-21T03:05:27.336 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feec00ad290 con 0x558c92aea1b0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feed2313740 10 client.25603 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558c92ed8e00 con 0x558c92aea1b0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.332+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7feec0020070 con 0x558c92aea1b0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 10 client.25603 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 10 client.25603 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 20 client.25603 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92ad19f0 0x558c92edc160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 4 client.25603 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558c92e32da0 con 0x558c92ad19f0 2024-09-21T03:05:27.337 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feec933a640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92ad19f0 0x558c92edc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feead7fa640 10 client.25603 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.333+0000 7feec933a640 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92ad19f0 0x558c92edc160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=41b8037f28fc49b8 server_cookie=c0f8468ed3e4f60 in_seq=0 out_seq=0 2024-09-21T03:05:27.338 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feead7fa640 1 -- 172.21.15.16:0/2077934032 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558c92e32da0 con 0x558c92ad19f0 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feead7fa640 10 client.25603 handle_command_reply: tid=0 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feed2313740 1 client.25603 shutdown 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 msgr2=0x558c92eca0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92eca0b0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7feec00d1000 tx=0x7feec00e7ff0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.334+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 shutdown_connections 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92ad19f0 0x558c92edc160 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92ecab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92af2390 0x558c92eca5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feead7fa640 0 client.25603 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 --2- 172.21.15.16:0/2077934032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92aea1b0 0x558c92eca0b0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 >> 172.21.15.16:0/2077934032 conn(0x558c92b07460 msgr2=0x558c92ebd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 shutdown_connections 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 1 -- 172.21.15.16:0/2077934032 wait complete. 2024-09-21T03:05:27.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.335+0000 7feed2313740 20 client.25603 trim_cache size 0 max 16384 2024-09-21T03:05:27.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.352+0000 7feed2313740 1 Processor -- start 2024-09-21T03:05:27.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 -- start start 2024-09-21T03:05:27.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92b13070 0x558c92eb7b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ab540 con 0x558c92af2390 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feec41addc0 con 0x558c92aea1b0 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feed2313740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec41abbb0 con 0x558c92b13070 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec933a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57006/0 (socket says 172.21.15.16:57006) 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feeca33c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92b13070 0x558c92eb7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.355 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 -- 172.21.15.16:0/463325085 learned_addr learned my addr 172.21.15.16:0/463325085 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 -- 172.21.15.16:0/463325085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92b13070 msgr2=0x558c92eb7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92b13070 0x558c92eb7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 -- 172.21.15.16:0/463325085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 msgr2=0x558c92ebde00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebde00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.353+0000 7feec9b3b640 1 -- 172.21.15.16:0/463325085 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558c92ebcd90 con 0x558c92aea1b0 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.354+0000 7feec9b3b640 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feec00aad90 tx=0x7feec00eb240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b96dcaa2c8973ed server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.354+0000 7feeaf7fe640 1 -- 172.21.15.16:0/463325085 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ca020 con 0x558c92aea1b0 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.354+0000 7feeaf7fe640 1 -- 172.21.15.16:0/463325085 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.354+0000 7feeaf7fe640 1 -- 172.21.15.16:0/463325085 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00ad8f0 con 0x558c92aea1b0 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- 172.21.15.16:0/463325085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 msgr2=0x558c92ebc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7feec00aad90 tx=0x7feec00eb240 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- 172.21.15.16:0/463325085 shutdown_connections 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92af2390 0x558c92ebde00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92b13070 0x558c92eb7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 --2- 172.21.15.16:0/463325085 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92ebc040 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.357 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- 172.21.15.16:0/463325085 >> 172.21.15.16:0/463325085 conn(0x558c92b07460 msgr2=0x558c92ebd6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- 172.21.15.16:0/463325085 shutdown_connections 2024-09-21T03:05:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- 172.21.15.16:0/463325085 wait complete. 2024-09-21T03:05:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 -- start start 2024-09-21T03:05:27.358 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.355+0000 7feed2313740 1 Processor -- start 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92af2390 0x558c92edd670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92eddbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ac280 con 0x558c92b13070 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ad6f0 con 0x558c92aea1b0 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feec41ac950 con 0x558c92af2390 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feed2313740 10 client.0 _ll_register_callbacks cb 0x7feeb400b690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:57016/0 (socket says 172.21.15.16:57016) 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 -- 172.21.15.16:0/497220011 learned_addr learned my addr 172.21.15.16:0/497220011 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec933a640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92af2390 0x558c92edd670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.361 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feeca33c640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92eddbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 -- 172.21.15.16:0/497220011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92af2390 msgr2=0x558c92edd670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92af2390 0x558c92edd670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 -- 172.21.15.16:0/497220011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 msgr2=0x558c92eddbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92eddbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.356+0000 7feec9b3b640 1 -- 172.21.15.16:0/497220011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558c92f458e0 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feec9b3b640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feec00ae240 tx=0x7feec00eb800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9bede933d11d7085 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feec00d5030 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7feec00b70b0 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7feec00ca070 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 10 client.25646 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558c92f458e0 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feec00aee50 con 0x558c92aea1b0 2024-09-21T03:05:27.362 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7feec0020070 con 0x558c92aea1b0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 10 client.25646 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 10 client.25646 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 20 client.25646 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92aa6da0 0x558c92f48e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 4 client.25646 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.357+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x558c92e2e850 con 0x558c92aa6da0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.358+0000 7feec933a640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92aa6da0 0x558c92f48e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.358+0000 7feead7fa640 10 client.25646 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.358+0000 7feec933a640 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92aa6da0 0x558c92f48e40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5ea0b7e598914226 server_cookie=e6a5912e652a4004 in_seq=0 out_seq=0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feead7fa640 1 -- 172.21.15.16:0/497220011 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2646 (crc 0 0 0) 0x558c92e2e850 con 0x558c92aa6da0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feead7fa640 10 client.25646 handle_command_reply: tid=0 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 client.25646 shutdown 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 msgr2=0x558c92edd130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7feec00ae240 tx=0x7feec00eb800 comp rx=0 tx=0).stop 2024-09-21T03:05:27.363 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 shutdown_connections 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558c92aa6da0 0x558c92f48e40 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558c92b13070 0x558c92eddbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558c92af2390 0x558c92edd670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feead7fa640 0 client.25646 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 --2- 172.21.15.16:0/497220011 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558c92aea1b0 0x558c92edd130 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 >> 172.21.15.16:0/497220011 conn(0x558c92b07460 msgr2=0x558c92ebbdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 shutdown_connections 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 1 -- 172.21.15.16:0/497220011 wait complete. 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.359+0000 7feed2313740 20 client.25646 trim_cache size 0 max 16384 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:05:27.364 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.365 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.366 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.367 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:05:27.368 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:05:27.369 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 msgr2=0x7feeb00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 0x7feeb00b8b90 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7feebc006010 tx=0x7feebc009460 comp rx=0 tx=0).stop 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 msgr2=0x7feec419a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7feec000ea60 tx=0x7feec000ef20 comp rx=0 tx=0).stop 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 shutdown_connections 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7feeb00b6680 0x7feeb00b8b90 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.370 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feec4108cf0 0x7feec419fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feec41083a0 0x7feec419aa90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 --2- 172.21.15.16:0/3754246222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feec4102a60 0x7feec419a4f0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 >> 172.21.15.16:0/3754246222 conn(0x7feec4079730 msgr2=0x7feec4105300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 shutdown_connections 2024-09-21T03:05:27.371 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:27.360+0000 7feecbd9f640 1 -- 172.21.15.16:0/3754246222 wait complete. 2024-09-21T03:05:28.247 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:05:28.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:05:28.744 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:05:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:28 smithi018 ceph-mon[34587]: pgmap v474: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 1.35k op/s 2024-09-21T03:05:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:28 smithi123 ceph-mon[34645]: pgmap v474: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 1.35k op/s 2024-09-21T03:05:29.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:28 smithi016 ceph-mon[31253]: pgmap v474: 97 pgs: 97 active+clean; 44 GiB data, 134 GiB used, 938 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 1.35k op/s 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.524+0000 7f195162a640 1 -- 172.21.15.16:0/897806303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105ad0 msgr2=0x7f194c10f560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.524+0000 7f195162a640 1 --2- 172.21.15.16:0/897806303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105ad0 0x7f194c10f560 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f1940009930 tx=0x7f194002a460 comp rx=0 tx=0).stop 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.526+0000 7f195162a640 1 -- 172.21.15.16:0/897806303 shutdown_connections 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.526+0000 7f195162a640 1 --2- 172.21.15.16:0/897806303 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c105180 0x7f194c105590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.527+0000 7f195162a640 1 --2- 172.21.15.16:0/897806303 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105ad0 0x7f194c10f560 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.527+0000 7f195162a640 1 --2- 172.21.15.16:0/897806303 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c103100 0x7f194c103510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.527+0000 7f195162a640 1 -- 172.21.15.16:0/897806303 >> 172.21.15.16:0/897806303 conn(0x7f194c077c60 msgr2=0x7f194c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.527+0000 7f195162a640 1 -- 172.21.15.16:0/897806303 shutdown_connections 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.527+0000 7f195162a640 1 -- 172.21.15.16:0/897806303 wait complete. 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.529+0000 7f195162a640 1 Processor -- start 2024-09-21T03:05:29.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.529+0000 7f195162a640 1 -- start start 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105180 0x7f194c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c105ad0 0x7f194c19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f194c111890 con 0x7f194c103100 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f194c111710 con 0x7f194c105ad0 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.531+0000 7f195162a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f194c111a10 con 0x7f194c105180 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.534+0000 7f194affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.534+0000 7f194affd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48406/0 (socket says 172.21.15.16:48406) 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.534+0000 7f194affd640 1 -- 172.21.15.16:0/2868685939 learned_addr learned my addr 172.21.15.16:0/2868685939 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.534+0000 7f194a7fc640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105180 0x7f194c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.534+0000 7f194b7fe640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c105ad0 0x7f194c19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.535+0000 7f194affd640 1 -- 172.21.15.16:0/2868685939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105180 msgr2=0x7f194c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.535+0000 7f194affd640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105180 0x7f194c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.535+0000 7f194affd640 1 -- 172.21.15.16:0/2868685939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c105ad0 msgr2=0x7f194c19da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:29.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.536+0000 7f194affd640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c105ad0 0x7f194c19da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.536+0000 7f194affd640 1 -- 172.21.15.16:0/2868685939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f194c19e190 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.536+0000 7f194affd640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f194400ebc0 tx=0x7f194400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.537+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944003b70 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.538+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1944003d10 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.538+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f194c19e480 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.538+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f194c1a4be0 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.538+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1944019800 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.541+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f19380052c0 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.549+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f19440199a0 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.550+0000 7f1933fff640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 0x7f19340bd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.551+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f19440e7f00 con 0x7f194c103100 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.551+0000 7f194a7fc640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 0x7f19340bd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.553+0000 7f194a7fc640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 0x7f19340bd170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f194c109230 tx=0x7f193c00b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:29.608 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:29.563+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f19440ae520 con 0x7f194c103100 2024-09-21T03:05:30.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.129+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f19380055b0 con 0x7f194c103100 2024-09-21T03:05:30.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.132+0000 7f1933fff640 1 -- 172.21.15.16:0/2868685939 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f19440b2900 con 0x7f194c103100 2024-09-21T03:05:30.135 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:05:30.135 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:05:30.135 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:05:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.144+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 msgr2=0x7f19340bd170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.144+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 0x7f19340bd170 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f194c109230 tx=0x7f193c00b390 comp rx=0 tx=0).stop 2024-09-21T03:05:30.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.144+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 msgr2=0x7f194c198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.144+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f194400ebc0 tx=0x7f194400c6a0 comp rx=0 tx=0).stop 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 shutdown_connections 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f19340bac60 0x7f19340bd170 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f194c105ad0 0x7f194c19da60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f194c105180 0x7f194c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 --2- 172.21.15.16:0/2868685939 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f194c103100 0x7f194c198220 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.145+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 >> 172.21.15.16:0/2868685939 conn(0x7f194c077c60 msgr2=0x7f194c106370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.146+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 shutdown_connections 2024-09-21T03:05:30.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:30.146+0000 7f195162a640 1 -- 172.21.15.16:0/2868685939 wait complete. 2024-09-21T03:05:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:30 smithi018 ceph-mon[34587]: pgmap v475: 97 pgs: 97 active+clean; 44 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 349 MiB/s wr, 871 op/s 2024-09-21T03:05:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2868685939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:05:31.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:30 smithi016 ceph-mon[31253]: pgmap v475: 97 pgs: 97 active+clean; 44 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 349 MiB/s wr, 871 op/s 2024-09-21T03:05:31.199 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2868685939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:05:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:30 smithi123 ceph-mon[34645]: pgmap v475: 97 pgs: 97 active+clean; 44 GiB data, 135 GiB used, 938 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 349 MiB/s wr, 871 op/s 2024-09-21T03:05:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2868685939' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:05:31.280 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:05:32.204 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.903+0000 7fab82d4e640 1 -- 172.21.15.16:0/894388202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c103bc0 msgr2=0x7fab7c103fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.903+0000 7fab82d4e640 1 --2- 172.21.15.16:0/894388202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c103bc0 0x7fab7c103fd0 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7fab70009900 tx=0x7fab7002a410 comp rx=0 tx=0).stop 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.904+0000 7fab82d4e640 1 -- 172.21.15.16:0/894388202 shutdown_connections 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.904+0000 7fab82d4e640 1 --2- 172.21.15.16:0/894388202 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c109e00 0x7fab7c110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.904+0000 7fab82d4e640 1 --2- 172.21.15.16:0/894388202 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab7c1094b0 0x7fab7c1098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.904+0000 7fab82d4e640 1 --2- 172.21.15.16:0/894388202 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c103bc0 0x7fab7c103fd0 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.904+0000 7fab82d4e640 1 -- 172.21.15.16:0/894388202 >> 172.21.15.16:0/894388202 conn(0x7fab7c07b910 msgr2=0x7fab7c07bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.907+0000 7fab82d4e640 1 -- 172.21.15.16:0/894388202 shutdown_connections 2024-09-21T03:05:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.908+0000 7fab82d4e640 1 -- 172.21.15.16:0/894388202 wait complete. 2024-09-21T03:05:32.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.909+0000 7fab82d4e640 1 Processor -- start 2024-09-21T03:05:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.910+0000 7fab82d4e640 1 -- start start 2024-09-21T03:05:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.910+0000 7fab82d4e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab7c103bc0 0x7fab7c19a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab82d4e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 0x7fab7c19ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab82d4e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 0x7fab7c19fe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab82d4e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113b80 con 0x7fab7c109e00 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab82d4e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113a00 con 0x7fab7c103bc0 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab82d4e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c113d00 con 0x7fab7c1094b0 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab812eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 0x7fab7c19fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab812eb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 0x7fab7c19fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48426/0 (socket says 172.21.15.16:48426) 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab812eb640 1 -- 172.21.15.16:0/4211584701 learned_addr learned my addr 172.21.15.16:0/4211584701 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:32.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.911+0000 7fab7bfff640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 0x7fab7c19ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.912+0000 7fab7bfff640 1 -- 172.21.15.16:0/4211584701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab7c103bc0 msgr2=0x7fab7c19a5f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.912+0000 7fab7bfff640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab7c103bc0 0x7fab7c19a5f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.912+0000 7fab7bfff640 1 -- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 msgr2=0x7fab7c19fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.912+0000 7fab7bfff640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 0x7fab7c19fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.912+0000 7fab7bfff640 1 -- 172.21.15.16:0/4211584701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab7c1a0560 con 0x7fab7c1094b0 2024-09-21T03:05:32.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.913+0000 7fab7bfff640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 0x7fab7c19ab90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab7c06bb30 tx=0x7fab6800b850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.914+0000 7fab79ffb640 1 -- 172.21.15.16:0/4211584701 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab6801d070 con 0x7fab7c1094b0 2024-09-21T03:05:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.914+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab7c1a0850 con 0x7fab7c1094b0 2024-09-21T03:05:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.914+0000 7fab79ffb640 1 -- 172.21.15.16:0/4211584701 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab68018440 con 0x7fab7c1094b0 2024-09-21T03:05:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.914+0000 7fab79ffb640 1 -- 172.21.15.16:0/4211584701 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab68016580 con 0x7fab7c1094b0 2024-09-21T03:05:32.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.914+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab7c075670 con 0x7fab7c1094b0 2024-09-21T03:05:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.918+0000 7fab892c2740 1 Processor -- start 2024-09-21T03:05:32.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.919+0000 7fab892c2740 1 -- start start 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a90001b0 0x55e4a93d0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9008390 0x55e4a93d0d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9346ea0 con 0x55e4a90001b0 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e4a93362d0 con 0x55e4a9008390 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.921+0000 7fab892c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4a933a0e0 con 0x55e4a9029070 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.922+0000 7fab812eb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.922+0000 7fab812eb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42842/0 (socket says 172.21.15.16:42842) 2024-09-21T03:05:32.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.922+0000 7fab812eb640 1 -- 172.21.15.16:0/968228320 learned_addr learned my addr 172.21.15.16:0/968228320 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.923+0000 7fab812eb640 1 -- 172.21.15.16:0/968228320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9008390 msgr2=0x55e4a93d0d40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:05:32.926 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.923+0000 7fab812eb640 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9008390 0x55e4a93d0d40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.923+0000 7fab812eb640 1 -- 172.21.15.16:0/968228320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a90001b0 msgr2=0x55e4a93d0800 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:05:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.924+0000 7fab812eb640 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a90001b0 0x55e4a93d0800 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.924+0000 7fab812eb640 1 -- 172.21.15.16:0/968228320 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a9094650 con 0x55e4a9029070 2024-09-21T03:05:32.927 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.925+0000 7fab79ffb640 1 -- 172.21.15.16:0/4211584701 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fab68020070 con 0x7fab7c1094b0 2024-09-21T03:05:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.931+0000 7fab812eb640 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e1c0 tx=0x7fab74009710 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2702a788a27ff245 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:32.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.932+0000 7fab79ffb640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.932+0000 7fab66ffd640 1 -- 172.21.15.16:0/968228320 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74018020 con 0x55e4a9029070 2024-09-21T03:05:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.932+0000 7fab66ffd640 1 -- 172.21.15.16:0/968228320 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74014b30 con 0x55e4a9029070 2024-09-21T03:05:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.932+0000 7fab66ffd640 1 -- 172.21.15.16:0/968228320 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74013b50 con 0x55e4a9029070 2024-09-21T03:05:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.932+0000 7fab79ffb640 1 -- 172.21.15.16:0/4211584701 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fab680ea6b0 con 0x7fab7c1094b0 2024-09-21T03:05:32.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.933+0000 7fab80aea640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 -- 172.21.15.16:0/968228320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 msgr2=0x55e4a93d02c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab80aea640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 0x7fab6c0b8c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab70009850 tx=0x7fab70005fd0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e1c0 tx=0x7fab74009710 comp rx=0 tx=0).stop 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 -- 172.21.15.16:0/968228320 shutdown_connections 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a90001b0 0x55e4a93d0800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9008390 0x55e4a93d0d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 --2- 172.21.15.16:0/968228320 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a9029070 0x55e4a93d02c0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.937 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 -- 172.21.15.16:0/968228320 >> 172.21.15.16:0/968228320 conn(0x55e4a901d460 msgr2=0x55e4a93ce1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.935+0000 7fab892c2740 1 -- 172.21.15.16:0/968228320 shutdown_connections 2024-09-21T03:05:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.936+0000 7fab892c2740 1 -- 172.21.15.16:0/968228320 wait complete. 2024-09-21T03:05:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.936+0000 7fab892c2740 1 -- start start 2024-09-21T03:05:32.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.936+0000 7fab892c2740 1 Processor -- start 2024-09-21T03:05:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93dbac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e4a9197700 con 0x55e4a9008390 2024-09-21T03:05:32.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e4a92d1160 con 0x55e4a9029070 2024-09-21T03:05:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e4a93deaa0 con 0x55e4a90001b0 2024-09-21T03:05:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.937+0000 7fab892c2740 10 client.0 _ll_register_callbacks cb 0x55e4a92ea9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.938+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.938+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48428/0 (socket says 172.21.15.16:48428) 2024-09-21T03:05:32.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.938+0000 7fab7bfff640 1 -- 172.21.15.16:0/2439332084 learned_addr learned my addr 172.21.15.16:0/2439332084 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab812eb640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab7bfff640 1 -- 172.21.15.16:0/2439332084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 msgr2=0x55e4a93dbac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93dbac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab7bfff640 1 -- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 msgr2=0x55e4a93e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.941 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.939+0000 7fab7bfff640 1 -- 172.21.15.16:0/2439332084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a92fec90 con 0x55e4a9008390 2024-09-21T03:05:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.941+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab680b0270 tx=0x7fab680aff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7a868b54446f523 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.941+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:05:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680e8250 con 0x55e4a9008390 2024-09-21T03:05:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fab680af9d0 con 0x55e4a9008390 2024-09-21T03:05:32.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab74014b30 con 0x55e4a9008390 2024-09-21T03:05:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fab680b4fa0 con 0x55e4a9008390 2024-09-21T03:05:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab892c2740 10 client.15984 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:32.945 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.942+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e4a93ea960 con 0x55e4a9008390 2024-09-21T03:05:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.944+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74014b30 con 0x55e4a9008390 2024-09-21T03:05:32.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.944+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fab680101e0 con 0x55e4a9008390 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 10 client.15984 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 10 client.15984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 20 client.15984 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a8fe79f0 0x55e4a93ede30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 4 client.15984 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:05:32.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.946+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e4a9348df0 con 0x55e4a8fe79f0 2024-09-21T03:05:32.955 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.953+0000 7fab80aea640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a8fe79f0 0x55e4a93ede30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.968+0000 7fab80aea640 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a8fe79f0 0x55e4a93ede30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=244d8b9eacdd1fda server_cookie=89fe53b6085cadb3 in_seq=0 out_seq=0 2024-09-21T03:05:32.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.969+0000 7fab5bfff640 10 client.15984 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab5bfff640 1 -- 172.21.15.16:0/2439332084 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e4a9348df0 con 0x55e4a8fe79f0 2024-09-21T03:05:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab5bfff640 10 client.15984 handle_command_reply: tid=0 2024-09-21T03:05:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 client.15984 shutdown 2024-09-21T03:05:32.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 msgr2=0x55e4a93dc040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7fab680b0270 tx=0x7fab680aff80 comp rx=0 tx=0).stop 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 shutdown_connections 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a8fe79f0 0x55e4a93ede30 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93dc040 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 --2- 172.21.15.16:0/2439332084 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93dbac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 >> 172.21.15.16:0/2439332084 conn(0x55e4a901d460 msgr2=0x55e4a93d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab5bfff640 0 client.15984 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:32.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 shutdown_connections 2024-09-21T03:05:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 1 -- 172.21.15.16:0/2439332084 wait complete. 2024-09-21T03:05:32.973 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.970+0000 7fab892c2740 20 client.15984 trim_cache size 0 max 16384 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.998+0000 7fab892c2740 1 Processor -- start 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.998+0000 7fab892c2740 1 -- start start 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93d8cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93d91f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93f3560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b3f10 con 0x55e4a9008390 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b6790 con 0x55e4a9029070 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab892c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b4580 con 0x55e4a90001b0 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab80aea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93d8cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab80aea640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93d8cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42868/0 (socket says 172.21.15.16:42868) 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:32.999+0000 7fab80aea640 1 -- 172.21.15.16:0/4160582128 learned_addr learned my addr 172.21.15.16:0/4160582128 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93f3560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 -- 172.21.15.16:0/4160582128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 msgr2=0x55e4a93d8cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93d8cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 -- 172.21.15.16:0/4160582128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 msgr2=0x55e4a93d91f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93d91f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.000+0000 7fab812eb640 1 -- 172.21.15.16:0/4160582128 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e4a93d27e0 con 0x55e4a9029070 2024-09-21T03:05:33.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.001+0000 7fab812eb640 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93f3560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e190 tx=0x7fab74014480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb05ca59a63bab1c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.001+0000 7fab667fc640 1 -- 172.21.15.16:0/4160582128 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab74018040 con 0x55e4a9029070 2024-09-21T03:05:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.001+0000 7fab667fc640 1 -- 172.21.15.16:0/4160582128 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab74014a20 con 0x55e4a9029070 2024-09-21T03:05:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 -- 172.21.15.16:0/4160582128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 msgr2=0x55e4a93f3560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93f3560 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fab7400e190 tx=0x7fab74014480 comp rx=0 tx=0).stop 2024-09-21T03:05:33.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 -- 172.21.15.16:0/4160582128 shutdown_connections 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93d91f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a9029070 0x55e4a93f3560 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 --2- 172.21.15.16:0/4160582128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93d8cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.003+0000 7fab892c2740 1 -- 172.21.15.16:0/4160582128 >> 172.21.15.16:0/4160582128 conn(0x55e4a901d460 msgr2=0x55e4a93d1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.004+0000 7fab892c2740 1 -- 172.21.15.16:0/4160582128 shutdown_connections 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.004+0000 7fab892c2740 1 -- 172.21.15.16:0/4160582128 wait complete. 2024-09-21T03:05:33.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.004+0000 7fab892c2740 1 -- start start 2024-09-21T03:05:33.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.005+0000 7fab892c2740 1 Processor -- start 2024-09-21T03:05:33.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93e1c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a8fbcda0 0x55e4a93e26f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b4c50 con 0x55e4a9008390 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b60c0 con 0x55e4a8fbcda0 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fab7c1b5320 con 0x55e4a90001b0 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.006+0000 7fab892c2740 10 client.0 _ll_register_callbacks cb 0x7fab680b0410 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.007+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.007+0000 7fab7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48438/0 (socket says 172.21.15.16:48438) 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.007+0000 7fab7bfff640 1 -- 172.21.15.16:0/2173805663 learned_addr learned my addr 172.21.15.16:0/2173805663 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:33.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.007+0000 7fab80aea640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93e1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 -- 172.21.15.16:0/2173805663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 msgr2=0x55e4a93e1c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93e1c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 -- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a8fbcda0 msgr2=0x55e4a93e26f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a8fbcda0 0x55e4a93e26f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 -- 172.21.15.16:0/2173805663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e4a9468780 con 0x55e4a9008390 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab7bfff640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab68000c00 tx=0x7fab680af680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=835d2a2fa586d285 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:33.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.008+0000 7fab5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:05:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.012+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab680e8250 con 0x55e4a9008390 2024-09-21T03:05:33.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.012+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fab68010220 con 0x55e4a9008390 2024-09-21T03:05:33.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.013+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55e4a93448a0 con 0x55e4a9008390 2024-09-21T03:05:33.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.015+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fab680acbc0 con 0x55e4a9008390 2024-09-21T03:05:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.015+0000 7fab892c2740 10 client.15987 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:33.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.015+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e4a9469a20 con 0x55e4a9008390 2024-09-21T03:05:33.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.018+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55e4a93448a0 con 0x55e4a9008390 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.018+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fab680ea110 con 0x55e4a9008390 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 10 client.15987 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 10 client.15987 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 20 client.15987 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a946a110 0x55e4a946c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 4 client.15987 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:05:33.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.019+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e4a9341cd0 con 0x55e4a946a110 2024-09-21T03:05:33.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.020+0000 7fab80aea640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a946a110 0x55e4a946c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:33.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.026+0000 7fab80aea640 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a946a110 0x55e4a946c530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72fd65a8e8469c0c server_cookie=eab4dcf5655d425a in_seq=0 out_seq=0 2024-09-21T03:05:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.028+0000 7fab5bfff640 10 client.15987 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.028+0000 7fab5bfff640 1 -- 172.21.15.16:0/2173805663 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e4a9341cd0 con 0x55e4a946a110 2024-09-21T03:05:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab5bfff640 10 client.15987 handle_command_reply: tid=0 2024-09-21T03:05:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 client.15987 shutdown 2024-09-21T03:05:33.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 msgr2=0x55e4a93e21b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fab68000c00 tx=0x7fab680af680 comp rx=0 tx=0).stop 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 shutdown_connections 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e4a946a110 0x55e4a946c530 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e4a9008390 0x55e4a93e21b0 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e4a8fbcda0 0x55e4a93e26f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 --2- 172.21.15.16:0/2173805663 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e4a90001b0 0x55e4a93e1c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 >> 172.21.15.16:0/2173805663 conn(0x55e4a901d460 msgr2=0x55e4a93dceb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.029+0000 7fab5bfff640 0 client.15987 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.030+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 shutdown_connections 2024-09-21T03:05:33.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.030+0000 7fab892c2740 1 -- 172.21.15.16:0/2173805663 wait complete. 2024-09-21T03:05:33.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.030+0000 7fab892c2740 20 client.15987 trim_cache size 0 max 16384 2024-09-21T03:05:33.033 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 msgr2=0x7fab6c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 0x7fab6c0b8c10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fab70009850 tx=0x7fab70005fd0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 msgr2=0x7fab7c19ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 0x7fab7c19ab90 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fab7c06bb30 tx=0x7fab6800b850 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 shutdown_connections 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fab6c0b6700 0x7fab6c0b8c10 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fab7c109e00 0x7fab7c19fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fab7c1094b0 0x7fab7c19ab90 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 --2- 172.21.15.16:0/4211584701 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fab7c103bc0 0x7fab7c19a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:33.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.032+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 >> 172.21.15.16:0/4211584701 conn(0x7fab7c07b910 msgr2=0x7fab7c107660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.041+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 shutdown_connections 2024-09-21T03:05:33.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:33.041+0000 7fab82d4e640 1 -- 172.21.15.16:0/4211584701 wait complete. 2024-09-21T03:05:33.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:32 smithi123 ceph-mon[34645]: pgmap v476: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 464 MiB/s wr, 830 op/s 2024-09-21T03:05:33.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:32 smithi016 ceph-mon[31253]: pgmap v476: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 464 MiB/s wr, 830 op/s 2024-09-21T03:05:33.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:32 smithi018 ceph-mon[34587]: pgmap v476: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 936 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 464 MiB/s wr, 830 op/s 2024-09-21T03:05:35.205 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:05:35.205 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:05:35.274 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:35 smithi016 ceph-mon[31253]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 433 MiB/s wr, 395 op/s 2024-09-21T03:05:35.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:35 smithi018 ceph-mon[34587]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 433 MiB/s wr, 395 op/s 2024-09-21T03:05:35.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:35 smithi123 ceph-mon[34645]: pgmap v477: 97 pgs: 97 active+clean; 45 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 433 MiB/s wr, 395 op/s 2024-09-21T03:05:36.136 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 -- 172.21.15.16:0/57191703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c111f40 msgr2=0x7f640c10a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 --2- 172.21.15.16:0/57191703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c111f40 0x7f640c10a3c0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f63fc009860 tx=0x7f63fc02a360 comp rx=0 tx=0).stop 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 -- 172.21.15.16:0/57191703 shutdown_connections 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 --2- 172.21.15.16:0/57191703 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c10adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 --2- 172.21.15.16:0/57191703 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c11f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 --2- 172.21.15.16:0/57191703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c111f40 0x7f640c10a3c0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:36.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.986+0000 7f6412060640 1 -- 172.21.15.16:0/57191703 >> 172.21.15.16:0/57191703 conn(0x7f640c070ec0 msgr2=0x7f640c0712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.997+0000 7f6412060640 1 -- 172.21.15.16:0/57191703 shutdown_connections 2024-09-21T03:05:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.997+0000 7f6412060640 1 -- 172.21.15.16:0/57191703 wait complete. 2024-09-21T03:05:37.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.998+0000 7f6412060640 1 Processor -- start 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.998+0000 7f6412060640 1 -- start start 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c1a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c1ae750 0x7f640c1aeb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f640c122960 con 0x7f640c10a9e0 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f640c1227e0 con 0x7f640c11cc30 2024-09-21T03:05:37.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:36.999+0000 7f6412060640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f640c122ae0 con 0x7f640c1ae750 2024-09-21T03:05:37.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.005+0000 7f640affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.005+0000 7f640affd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55446/0 (socket says 172.21.15.16:55446) 2024-09-21T03:05:37.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.005+0000 7f640affd640 1 -- 172.21.15.16:0/3947327965 learned_addr learned my addr 172.21.15.16:0/3947327965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:37.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.006+0000 7f640b7fe640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c1a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640bfff640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c1ae750 0x7f640c1aeb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640affd640 1 -- 172.21.15.16:0/3947327965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c1ae750 msgr2=0x7f640c1aeb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640affd640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c1ae750 0x7f640c1aeb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640affd640 1 -- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 msgr2=0x7f640c1a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640affd640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c1a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640affd640 1 -- 172.21.15.16:0/3947327965 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f640c1af3a0 con 0x7f640c11cc30 2024-09-21T03:05:37.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.007+0000 7f640b7fe640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c1a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:37.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.009+0000 7f640affd640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f640000b450 tx=0x7f640000b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.010+0000 7f6408ff9640 1 -- 172.21.15.16:0/3947327965 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400015020 con 0x7f640c11cc30 2024-09-21T03:05:37.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.011+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f640c112380 con 0x7f640c11cc30 2024-09-21T03:05:37.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.011+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f640c1128c0 con 0x7f640c11cc30 2024-09-21T03:05:37.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.011+0000 7f6408ff9640 1 -- 172.21.15.16:0/3947327965 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6400011d40 con 0x7f640c11cc30 2024-09-21T03:05:37.015 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.011+0000 7f6408ff9640 1 -- 172.21.15.16:0/3947327965 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f640000f340 con 0x7f640c11cc30 2024-09-21T03:05:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.021+0000 7f6408ff9640 1 -- 172.21.15.16:0/3947327965 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6400018020 con 0x7f640c11cc30 2024-09-21T03:05:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.023+0000 7f6408ff9640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 0x7f63f40b8c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.024+0000 7f6408ff9640 1 -- 172.21.15.16:0/3947327965 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f64000e6fa0 con 0x7f640c11cc30 2024-09-21T03:05:37.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.027+0000 7f640b7fe640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 0x7f63f40b8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.029+0000 7f64185d4740 1 Processor -- start 2024-09-21T03:05:37.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.030+0000 7f64185d4740 1 -- start start 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.030+0000 7f64185d4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e702a350 0x55f6e73d1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.030+0000 7f64185d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.031+0000 7f64185d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.031+0000 7f64185d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7347e60 con 0x55f6e7009670 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.031+0000 7f64185d4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7337290 con 0x55f6e702a350 2024-09-21T03:05:37.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.031+0000 7f64185d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6e733b0a0 con 0x55f6e7001490 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.032+0000 7f640affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.032+0000 7f640affd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39370/0 (socket says 172.21.15.16:39370) 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.032+0000 7f640affd640 1 -- 172.21.15.16:0/1325558152 learned_addr learned my addr 172.21.15.16:0/1325558152 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.032+0000 7f640b7fe640 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e702a350 0x55f6e73d1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.032+0000 7f640b7fe640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 0x7f63f40b8c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f63fc000c00 tx=0x7f63fc009280 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 -- 172.21.15.16:0/1325558152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e702a350 msgr2=0x55f6e73d1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e702a350 0x55f6e73d1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 -- 172.21.15.16:0/1325558152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 msgr2=0x55f6e73d1eb0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:05:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d1eb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 -- 172.21.15.16:0/1325558152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e7095930 con 0x55f6e7001490 2024-09-21T03:05:37.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.033+0000 7f640affd640 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64000acab0 tx=0x7f6400010fc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7ba8a8630ced1cb4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.034+0000 7f63f1ffb640 1 -- 172.21.15.16:0/1325558152 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64000b67d0 con 0x55f6e7001490 2024-09-21T03:05:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.034+0000 7f63f1ffb640 1 -- 172.21.15.16:0/1325558152 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64000b0020 con 0x55f6e7001490 2024-09-21T03:05:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.034+0000 7f63f1ffb640 1 -- 172.21.15.16:0/1325558152 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6400092b80 con 0x55f6e7001490 2024-09-21T03:05:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.035+0000 7f64185d4740 1 -- 172.21.15.16:0/1325558152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 msgr2=0x55f6e73d1970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.035+0000 7f64185d4740 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f64000acab0 tx=0x7f6400010fc0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 -- 172.21.15.16:0/1325558152 shutdown_connections 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d1eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e7001490 0x55f6e73d1970 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 --2- 172.21.15.16:0/1325558152 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e702a350 0x55f6e73d1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 -- 172.21.15.16:0/1325558152 >> 172.21.15.16:0/1325558152 conn(0x55f6e701e740 msgr2=0x55f6e73cf320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 -- 172.21.15.16:0/1325558152 shutdown_connections 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.037+0000 7f64185d4740 1 -- 172.21.15.16:0/1325558152 wait complete. 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.038+0000 7f64185d4740 1 -- start start 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.038+0000 7f64185d4740 1 Processor -- start 2024-09-21T03:05:37.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.040+0000 7f64185d4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.040+0000 7f64185d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73dd650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.041+0000 7f64185d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73e8f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.041+0000 7f64185d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55f6e7198890 con 0x55f6e7009670 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.041+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.041+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55466/0 (socket says 172.21.15.16:55466) 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.041+0000 7f640b7fe640 1 -- 172.21.15.16:0/2188290536 learned_addr learned my addr 172.21.15.16:0/2188290536 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:37.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.042+0000 7f640bfff640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73e8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.042+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55f6e72d1d40 con 0x55f6e7001490 2024-09-21T03:05:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.042+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55f6e73dfc10 con 0x55f6e702a350 2024-09-21T03:05:37.044 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.042+0000 7f64185d4740 10 client.0 _ll_register_callbacks cb 0x55f6e7335c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.043+0000 7f640b7fe640 1 -- 172.21.15.16:0/2188290536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 msgr2=0x55f6e73e8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.043+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73e8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.043+0000 7f640b7fe640 1 -- 172.21.15.16:0/2188290536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 msgr2=0x55f6e73dd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.043+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73dd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.043+0000 7f640b7fe640 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e73e95a0 con 0x55f6e7001490 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.047+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0308f0 tx=0x7f63fc005f90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fb5bed1fb2edbf38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.049 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.047+0000 7f63eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:05:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.048+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc04c020 con 0x55f6e7001490 2024-09-21T03:05:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.048+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64000b0020 con 0x55f6e7001490 2024-09-21T03:05:37.051 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.049+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f63fc0046c0 con 0x55f6e7001490 2024-09-21T03:05:37.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.069+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63fc057050 con 0x55f6e7001490 2024-09-21T03:05:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.069+0000 7f64185d4740 10 client.25667 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:37.072 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.069+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f6e73ebf10 con 0x55f6e7001490 2024-09-21T03:05:37.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.070+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64000b0020 con 0x55f6e7001490 2024-09-21T03:05:37.073 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f63fc003a70 con 0x55f6e7001490 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 10 client.25667 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 10 client.25667 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 20 client.25667 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e6fe8cd0 0x55f6e73ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 4 client.25667 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.071+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f6e7349db0 con 0x55f6e6fe8cd0 2024-09-21T03:05:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.072+0000 7f640affd640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e6fe8cd0 0x55f6e73ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.073+0000 7f640affd640 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e6fe8cd0 0x55f6e73ef8f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=711960702db0eca1 server_cookie=2ee5e394b1e979d4 in_seq=0 out_seq=0 2024-09-21T03:05:37.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.073+0000 7f63eaffd640 10 client.25667 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.074+0000 7f63eaffd640 1 -- 172.21.15.16:0/2188290536 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55f6e7349db0 con 0x55f6e6fe8cd0 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.074+0000 7f63eaffd640 10 client.25667 handle_command_reply: tid=0 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 client.25667 shutdown 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 msgr2=0x55f6e73dd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0308f0 tx=0x7f63fc005f90 comp rx=0 tx=0).stop 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 shutdown_connections 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e6fe8cd0 0x55f6e73ef8f0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73dd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73e8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 --2- 172.21.15.16:0/2188290536 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73dd0b0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.075+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 >> 172.21.15.16:0/2188290536 conn(0x55f6e701e740 msgr2=0x55f6e73d45c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.076+0000 7f63eaffd640 0 client.25667 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.076+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 shutdown_connections 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.076+0000 7f64185d4740 1 -- 172.21.15.16:0/2188290536 wait complete. 2024-09-21T03:05:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.076+0000 7f64185d4740 20 client.25667 trim_cache size 0 max 16384 2024-09-21T03:05:37.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.119+0000 7f64185d4740 1 Processor -- start 2024-09-21T03:05:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.119+0000 7f64185d4740 1 -- start start 2024-09-21T03:05:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.119+0000 7f64185d4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73d4530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.120+0000 7f64185d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.120+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73d4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.121+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73d4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55472/0 (socket says 172.21.15.16:55472) 2024-09-21T03:05:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.121+0000 7f640b7fe640 1 -- 172.21.15.16:0/4197404408 learned_addr learned my addr 172.21.15.16:0/4197404408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:37.123 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.122+0000 7f64185d4740 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73d64c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.122+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800c6f0 con 0x55f6e7009670 2024-09-21T03:05:37.124 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.122+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800eec0 con 0x55f6e7001490 2024-09-21T03:05:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.123+0000 7f640bfff640 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73d64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.125 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.123+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800ccb0 con 0x55f6e702a350 2024-09-21T03:05:37.128 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.126+0000 7f640affd640 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.127+0000 7f640bfff640 1 -- 172.21.15.16:0/4197404408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 msgr2=0x55f6e73d4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.127+0000 7f640bfff640 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73d4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.127+0000 7f640bfff640 1 -- 172.21.15.16:0/4197404408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 msgr2=0x55f6e73d4a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.127+0000 7f640bfff640 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d4a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.127+0000 7f640bfff640 1 -- 172.21.15.16:0/4197404408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6e73cf860 con 0x55f6e702a350 2024-09-21T03:05:37.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.129+0000 7f640bfff640 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73d64c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6404009f60 tx=0x7f6404009f90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d4b0710944dbf3e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.130+0000 7f63f17fa640 1 -- 172.21.15.16:0/4197404408 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6404004b20 con 0x55f6e702a350 2024-09-21T03:05:37.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.130+0000 7f63f17fa640 1 -- 172.21.15.16:0/4197404408 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6404003e20 con 0x55f6e702a350 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 msgr2=0x55f6e73d64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73d64c0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f6404009f60 tx=0x7f6404009f90 comp rx=0 tx=0).stop 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 shutdown_connections 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73d4a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e702a350 0x55f6e73d64c0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 --2- 172.21.15.16:0/4197404408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73d4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.131+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 >> 172.21.15.16:0/4197404408 conn(0x55f6e701e740 msgr2=0x55f6e73db730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.132+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 shutdown_connections 2024-09-21T03:05:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.132+0000 7f64185d4740 1 -- 172.21.15.16:0/4197404408 wait complete. 2024-09-21T03:05:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.132+0000 7f64185d4740 1 -- start start 2024-09-21T03:05:37.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 Processor -- start 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73e1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e6fbe080 0x55f6e73e1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800d380 con 0x55f6e7009670 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800e7f0 con 0x55f6e7001490 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f63f800da50 con 0x55f6e6fbe080 2024-09-21T03:05:37.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.133+0000 7f64185d4740 10 client.0 _ll_register_callbacks cb 0x7f63fc0096b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:37.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.134+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.136+0000 7f640b7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55480/0 (socket says 172.21.15.16:55480) 2024-09-21T03:05:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.136+0000 7f640b7fe640 1 -- 172.21.15.16:0/2195649223 learned_addr learned my addr 172.21.15.16:0/2195649223 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:37.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.136+0000 7f640bfff640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e6fbe080 0x55f6e73e1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.142+0000 7f640affd640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73e1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640b7fe640 1 -- 172.21.15.16:0/2195649223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e6fbe080 msgr2=0x55f6e73e1970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e6fbe080 0x55f6e73e1970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640b7fe640 1 -- 172.21.15.16:0/2195649223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 msgr2=0x55f6e73e1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73e1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640b7fe640 1 -- 172.21.15.16:0/2195649223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f6e7459340 con 0x55f6e7001490 2024-09-21T03:05:37.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.143+0000 7f640affd640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:05:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.144+0000 7f640b7fe640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0308f0 tx=0x7f63fc005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4281a27f0fa14087 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:37.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.144+0000 7f63eaffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:05:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.145+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63fc04c040 con 0x55f6e7001490 2024-09-21T03:05:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.145+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f63fc003a00 con 0x55f6e7001490 2024-09-21T03:05:37.147 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.146+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55f6e7345860 con 0x55f6e7001490 2024-09-21T03:05:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.146+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63fc042040 con 0x55f6e7001490 2024-09-21T03:05:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.146+0000 7f64185d4740 10 client.25673 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:37.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.146+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55f6e745a6c0 con 0x55f6e7001490 2024-09-21T03:05:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.155+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55f6e7345860 con 0x55f6e7001490 2024-09-21T03:05:37.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.155+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f63fc063030 con 0x55f6e7001490 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.155+0000 7f64185d4740 10 client.25673 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.155+0000 7f64185d4740 10 client.25673 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.155+0000 7f64185d4740 20 client.25673 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.156+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e745aea0 0x55f6e745d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.156+0000 7f64185d4740 4 client.25673 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:05:37.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.156+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55f6e7342c90 con 0x55f6e745aea0 2024-09-21T03:05:37.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.158+0000 7f640affd640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e745aea0 0x55f6e745d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.161+0000 7f640affd640 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e745aea0 0x55f6e745d2c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=629e82639e8c48aa server_cookie=59149266c6de68bf in_seq=0 out_seq=0 2024-09-21T03:05:37.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.161+0000 7f63eaffd640 10 client.25673 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.164+0000 7f63eaffd640 1 -- 172.21.15.16:0/2195649223 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55f6e7342c90 con 0x55f6e745aea0 2024-09-21T03:05:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.164+0000 7f63eaffd640 10 client.25673 handle_command_reply: tid=0 2024-09-21T03:05:37.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.164+0000 7f64185d4740 1 client.25673 shutdown 2024-09-21T03:05:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.164+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 msgr2=0x55f6e73e0ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.164+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f63fc0308f0 tx=0x7f63fc005640 comp rx=0 tx=0).stop 2024-09-21T03:05:37.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 shutdown_connections 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55f6e745aea0 0x55f6e745d2c0 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55f6e7009670 0x55f6e73e1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f6e6fbe080 0x55f6e73e1970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 --2- 172.21.15.16:0/2195649223 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55f6e7001490 0x55f6e73e0ef0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 >> 172.21.15.16:0/2195649223 conn(0x55f6e701e740 msgr2=0x55f6e73d4530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f63eaffd640 0 client.25673 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 shutdown_connections 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 1 -- 172.21.15.16:0/2195649223 wait complete. 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:05:37.167 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:05:37.168 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "d4145247-6bd4-4fe4-a89f-639a0a34ac56", 2024-09-21T03:05:37.168 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:05:37.168 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:05:37.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.165+0000 7f64185d4740 20 client.25673 trim_cache size 0 max 16384 2024-09-21T03:05:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 msgr2=0x7f63f40b8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 0x7f63f40b8c40 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f63fc000c00 tx=0x7f63fc009280 comp rx=0 tx=0).stop 2024-09-21T03:05:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 msgr2=0x7f640c1a9570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f640000b450 tx=0x7f640000b910 comp rx=0 tx=0).stop 2024-09-21T03:05:37.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 shutdown_connections 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63f40b6730 0x7f63f40b8c40 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f640c1ae750 0x7f640c1aeb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f640c11cc30 0x7f640c1a9570 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 --2- 172.21.15.16:0/3947327965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f640c10a9e0 0x7f640c1a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 >> 172.21.15.16:0/3947327965 conn(0x7f640c070ec0 msgr2=0x7f640c071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 shutdown_connections 2024-09-21T03:05:37.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:37.168+0000 7f6412060640 1 -- 172.21.15.16:0/3947327965 wait complete. 2024-09-21T03:05:37.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:37 smithi018 ceph-mon[34587]: pgmap v478: 97 pgs: 97 active+clean; 48 GiB data, 146 GiB used, 927 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 651 MiB/s wr, 866 op/s 2024-09-21T03:05:37.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:37 smithi123 ceph-mon[34645]: pgmap v478: 97 pgs: 97 active+clean; 48 GiB data, 146 GiB used, 927 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 651 MiB/s wr, 866 op/s 2024-09-21T03:05:37.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:37 smithi016 ceph-mon[31253]: pgmap v478: 97 pgs: 97 active+clean; 48 GiB data, 146 GiB used, 927 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 651 MiB/s wr, 866 op/s 2024-09-21T03:05:37.964 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:05:38.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:38 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:05:38.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:38 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:38.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:38 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:05:38.333 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:38 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:05:38.333 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:38 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:38.333 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:38 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:05:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:38 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:05:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:38 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:38 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:05:38.553 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:05:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:39 smithi016 ceph-mon[31253]: pgmap v479: 97 pgs: 97 active+clean; 49 GiB data, 148 GiB used, 925 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 543 MiB/s wr, 873 op/s 2024-09-21T03:05:39.360 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:39 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:39.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:39 smithi018 ceph-mon[34587]: pgmap v479: 97 pgs: 97 active+clean; 49 GiB data, 148 GiB used, 925 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 543 MiB/s wr, 873 op/s 2024-09-21T03:05:39.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:39 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:39 smithi123 ceph-mon[34645]: pgmap v479: 97 pgs: 97 active+clean; 49 GiB data, 148 GiB used, 925 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 543 MiB/s wr, 873 op/s 2024-09-21T03:05:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:39 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.469+0000 7fb383fbb640 1 -- 172.21.15.16:0/4190365575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c108cf0 msgr2=0x7fb37c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.469+0000 7fb383fbb640 1 --2- 172.21.15.16:0/4190365575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c108cf0 0x7fb37c10f520 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fb374009be0 tx=0x7fb37402a6c0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.469+0000 7fb383fbb640 1 -- 172.21.15.16:0/4190365575 shutdown_connections 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.469+0000 7fb383fbb640 1 --2- 172.21.15.16:0/4190365575 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c108cf0 0x7fb37c10f520 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.470+0000 7fb383fbb640 1 --2- 172.21.15.16:0/4190365575 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.470+0000 7fb383fbb640 1 --2- 172.21.15.16:0/4190365575 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c102a60 0x7fb37c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.470+0000 7fb383fbb640 1 -- 172.21.15.16:0/4190365575 >> 172.21.15.16:0/4190365575 conn(0x7fb37c079730 msgr2=0x7fb37c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.470+0000 7fb383fbb640 1 -- 172.21.15.16:0/4190365575 shutdown_connections 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.470+0000 7fb383fbb640 1 -- 172.21.15.16:0/4190365575 wait complete. 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 Processor -- start 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 -- start start 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.577 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 0x7fb37c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.471+0000 7fb383fbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c112960 con 0x7fb37c108cf0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb383fbb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1127e0 con 0x7fb37c1083a0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb383fbb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c112ae0 con 0x7fb37c102a60 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb382558640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 0x7fb37c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381d57640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55490/0 (socket says 172.21.15.16:55490) 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381d57640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39422/0 (socket says 172.21.15.16:39422) 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 -- 172.21.15.16:0/742142014 learned_addr learned my addr 172.21.15.16:0/742142014 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 -- 172.21.15.16:0/742142014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 msgr2=0x7fb37c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 -- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 msgr2=0x7fb37c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 0x7fb37c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 -- 172.21.15.16:0/742142014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb37c1a0350 con 0x7fb37c1083a0 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb382558640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 0x7fb37c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:39.578 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381d57640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.472+0000 7fb381556640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00e9c0 tx=0x7fb36c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.473+0000 7fb372ffd640 1 -- 172.21.15.16:0/742142014 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c00cd80 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.473+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb37c1a0640 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.473+0000 7fb372ffd640 1 -- 172.21.15.16:0/742142014 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c0054d0 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.473+0000 7fb372ffd640 1 -- 172.21.15.16:0/742142014 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c010430 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.473+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb37c1a6da0 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.476+0000 7fb372ffd640 1 -- 172.21.15.16:0/742142014 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb36c0105d0 con 0x7fb37c1083a0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.477+0000 7fb38a52f740 1 Processor -- start 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.477+0000 7fb38a52f740 1 -- start start 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 0x55ca39c5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb381d57640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb382558640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 0x55ca39c5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb381d57640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39436/0 (socket says 172.21.15.16:39436) 2024-09-21T03:05:39.579 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb382558640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 0x55ca39c5a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55500/0 (socket says 172.21.15.16:55500) 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c5b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb381d57640 1 -- 172.21.15.16:0/3565662965 learned_addr learned my addr 172.21.15.16:0/3565662965 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca39bd0e50 con 0x55ca39892390 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ca39bc0280 con 0x55ca398b3070 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca39bc4090 con 0x55ca3988a1b0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb372ffd640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 0x7fb3680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.478+0000 7fb381556640 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c5b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb382558640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 0x7fb3680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb372ffd640 1 -- 172.21.15.16:0/742142014 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb36c0e7b00 con 0x7fb37c1083a0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381d57640 1 -- 172.21.15.16:0/3565662965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 msgr2=0x55ca39c5a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381d57640 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 0x55ca39c5a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381d57640 1 -- 172.21.15.16:0/3565662965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 msgr2=0x55ca39c5b0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381d57640 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c5b0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381d57640 1 -- 172.21.15.16:0/3565662965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca3991e650 con 0x55ca3988a1b0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.479+0000 7fb381556640 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c5b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.480+0000 7fb381d57640 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3780114f0 tx=0x7fb3780119b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c544a2ab2517e144 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.580 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.480+0000 7fb382558640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 0x7fb3680b8c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb374009370 tx=0x7fb37402a6c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.480+0000 7fb367fff640 1 -- 172.21.15.16:0/3565662965 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378005020 con 0x55ca3988a1b0 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.480+0000 7fb367fff640 1 -- 172.21.15.16:0/3565662965 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3780047a0 con 0x55ca3988a1b0 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.480+0000 7fb367fff640 1 -- 172.21.15.16:0/3565662965 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb37800f860 con 0x55ca3988a1b0 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 msgr2=0x55ca39c5ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fb3780114f0 tx=0x7fb3780119b0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 shutdown_connections 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c5b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c5ab90 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3565662965 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398b3070 0x55ca39c5a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 >> 172.21.15.16:0/3565662965 conn(0x55ca398a7460 msgr2=0x55ca39c58540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 shutdown_connections 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.482+0000 7fb38a52f740 1 -- 172.21.15.16:0/3565662965 wait complete. 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.483+0000 7fb38a52f740 1 -- start start 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.483+0000 7fb38a52f740 1 Processor -- start 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.581 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c66230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb381d57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca398b3070 0x55ca39c72150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb381556640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c66230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb381d57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55502/0 (socket says 172.21.15.16:55502) 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb381d57640 1 -- 172.21.15.16:0/3141510691 learned_addr learned my addr 172.21.15.16:0/3141510691 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55ca39a21700 con 0x55ca39892390 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.484+0000 7fb382558640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca398b3070 0x55ca39c72150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55ca39b5b110 con 0x55ca3988a1b0 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55ca39c68e30 con 0x55ca398b3070 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb38a52f740 10 client.0 _ll_register_callbacks cb 0x55ca39b74990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381d57640 1 -- 172.21.15.16:0/3141510691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca398b3070 msgr2=0x55ca39c72150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381d57640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca398b3070 0x55ca39c72150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381d57640 1 -- 172.21.15.16:0/3141510691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 msgr2=0x55ca39c66230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381d57640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c66230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.582 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381d57640 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca39b88c40 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.485+0000 7fb381556640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb381d57640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb37801b040 tx=0x7fb378005110 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7a8d7b07d323fb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb3657fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb378028070 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb378005490 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3780047a0 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb378033050 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.486+0000 7fb38a52f740 10 client.25685 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ca39c74b10 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3780047a0 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb378003c40 con 0x55ca3988a1b0 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 10 client.25685 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 10 client.25685 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 20 client.25685 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398719f0 0x55ca39c77ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.583 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 4 client.25685 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.487+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ca39bd2da0 con 0x55ca398719f0 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.488+0000 7fb381556640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398719f0 0x55ca39c77ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.488+0000 7fb381556640 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398719f0 0x55ca39c77ed0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b61765489ba3222 server_cookie=212ce88311f6e432 in_seq=0 out_seq=0 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.488+0000 7fb3657fa640 10 client.25685 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.489+0000 7fb3657fa640 1 -- 172.21.15.16:0/3141510691 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55ca39bd2da0 con 0x55ca398719f0 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.489+0000 7fb3657fa640 10 client.25685 handle_command_reply: tid=0 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.489+0000 7fb38a52f740 1 client.25685 shutdown 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.489+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 msgr2=0x55ca39c65cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.489+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fb37801b040 tx=0x7fb378005110 comp rx=0 tx=0).stop 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 shutdown_connections 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398719f0 0x55ca39c77ed0 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c66230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca398b3070 0x55ca39c72150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb3657fa640 0 client.25685 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:39.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 --2- 172.21.15.16:0/3141510691 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c65cf0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 >> 172.21.15.16:0/3141510691 conn(0x55ca398a7460 msgr2=0x55ca39c5d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 shutdown_connections 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 1 -- 172.21.15.16:0/3141510691 wait complete. 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.490+0000 7fb38a52f740 20 client.25685 trim_cache size 0 max 16384 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 Processor -- start 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 -- start start 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 0x55ca39c67bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 0x55ca39c57b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ad6f0 con 0x55ca398719f0 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aff70 con 0x55ca3988a1b0 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.525+0000 7fb38a52f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1add60 con 0x55ca39892390 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55506/0 (socket says 172.21.15.16:55506) 2024-09-21T03:05:39.585 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 -- 172.21.15.16:0/1685629945 learned_addr learned my addr 172.21.15.16:0/1685629945 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381d57640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 0x55ca39c67bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb382558640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 0x55ca39c57b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 -- 172.21.15.16:0/1685629945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 msgr2=0x55ca39c57b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 0x55ca39c57b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 -- 172.21.15.16:0/1685629945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 msgr2=0x55ca39c67bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 0x55ca39c67bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381556640 1 -- 172.21.15.16:0/1685629945 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ca39c5d4b0 con 0x55ca3988a1b0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb381d57640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 0x55ca39c67bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.526+0000 7fb382558640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 0x55ca39c57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.527+0000 7fb381556640 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0e5870 tx=0x7fb36c0d1040 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4feac1bcbb00ce8f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.527+0000 7fb3677fe640 1 -- 172.21.15.16:0/1685629945 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0046c0 con 0x55ca3988a1b0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.527+0000 7fb3677fe640 1 -- 172.21.15.16:0/1685629945 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c0ad480 con 0x55ca3988a1b0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.527+0000 7fb3677fe640 1 -- 172.21.15.16:0/1685629945 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b7100 con 0x55ca3988a1b0 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- 172.21.15.16:0/1685629945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 msgr2=0x55ca39c5bd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0e5870 tx=0x7fb36c0d1040 comp rx=0 tx=0).stop 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- 172.21.15.16:0/1685629945 shutdown_connections 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca39892390 0x55ca39c57b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca3988a1b0 0x55ca39c5bd90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 --2- 172.21.15.16:0/1685629945 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca398719f0 0x55ca39c67bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- 172.21.15.16:0/1685629945 >> 172.21.15.16:0/1685629945 conn(0x55ca398a7460 msgr2=0x55ca39c5b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- 172.21.15.16:0/1685629945 shutdown_connections 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- 172.21.15.16:0/1685629945 wait complete. 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 -- start start 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.528+0000 7fb38a52f740 1 Processor -- start 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 0x55ca39c6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c69f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1ae430 con 0x55ca39892390 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1af8a0 con 0x55ca398719f0 2024-09-21T03:05:39.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb37c1aeb00 con 0x55ca3988a1b0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381556640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381d57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 0x55ca39c6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381556640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39462/0 (socket says 172.21.15.16:39462) 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381d57640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 0x55ca39c6e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55514/0 (socket says 172.21.15.16:55514) 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381556640 1 -- 172.21.15.16:0/246631260 learned_addr learned my addr 172.21.15.16:0/246631260 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb38a52f740 10 client.0 _ll_register_callbacks cb 0x7fb36c0a9810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb382558640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c69f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381556640 1 -- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 msgr2=0x55ca39c6e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.529+0000 7fb381556640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 0x55ca39c6e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb381556640 1 -- 172.21.15.16:0/246631260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 msgr2=0x55ca39c69f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb381556640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c69f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb381556640 1 -- 172.21.15.16:0/246631260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ca39c6a470 con 0x55ca3988a1b0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb382558640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb381556640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0a9250 tx=0x7fb36c0a9940 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d06527e480aadbcd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb3657fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb36c0b7100 con 0x55ca3988a1b0 2024-09-21T03:05:39.588 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7fb36c0acbd0 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb36c0ad480 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb36c0046c0 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb38a52f740 10 client.25657 fetch_fsmap learned FSMap version 20 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55ca39ce2320 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.530+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb36c0ad480 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb36c0d2040 con 0x55ca3988a1b0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 10 client.25657 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 10 client.25657 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 20 client.25657 populate_metadata read hostname 'smithi016' 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398b3070 0x55ca39ce4a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 4 client.25657 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ca39c665e0 con 0x55ca398b3070 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.531+0000 7fb381d57640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398b3070 0x55ca39ce4a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb3657fa640 10 client.25657 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb381d57640 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398b3070 0x55ca39ce4a70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=121359c6a088db12 server_cookie=1ab7114766c2f239 in_seq=0 out_seq=0 2024-09-21T03:05:39.589 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb3657fa640 1 -- 172.21.15.16:0/246631260 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3126 (crc 0 0 0) 0x55ca39c665e0 con 0x55ca398b3070 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb3657fa640 10 client.25657 handle_command_reply: tid=0 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 client.25657 shutdown 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 msgr2=0x55ca39c699f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fb36c0a9250 tx=0x7fb36c0a9940 comp rx=0 tx=0).stop 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "d4145247-6bd4-4fe4-a89f-639a0a34ac56", 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:05:39.590 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.591 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.592 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.593 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.594 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:05:39.595 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 shutdown_connections 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55ca398b3070 0x55ca39ce4a70 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55ca39892390 0x55ca39c69f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55ca3988a1b0 0x55ca39c699f0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 --2- 172.21.15.16:0/246631260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55ca398719f0 0x55ca39c6e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb3657fa640 0 client.25657 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 >> 172.21.15.16:0/246631260 conn(0x55ca398a7460 msgr2=0x55ca39c67930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.596 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 shutdown_connections 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 1 -- 172.21.15.16:0/246631260 wait complete. 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.532+0000 7fb38a52f740 20 client.25657 trim_cache size 0 max 16384 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 msgr2=0x7fb3680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 0x7fb3680b8c10 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fb374009370 tx=0x7fb37402a6c0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 msgr2=0x7fb37c19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fb36c00e9c0 tx=0x7fb36c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 shutdown_connections 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb3680b6700 0x7fb3680b8c10 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb37c108cf0 0x7fb37c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb37c1083a0 0x7fb37c19a980 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 --2- 172.21.15.16:0/742142014 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb37c102a60 0x7fb37c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 >> 172.21.15.16:0/742142014 conn(0x7fb37c079730 msgr2=0x7fb37c06b1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.533+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 shutdown_connections 2024-09-21T03:05:39.597 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:05:39.534+0000 7fb383fbb640 1 -- 172.21.15.16:0/742142014 wait complete. 2024-09-21T03:05:40.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:40 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:05:40.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:40 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:05:40.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:40 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:05:40.758 INFO:tasks.cephfs.filesystem:scrub status for tag:d4145247-6bd4-4fe4-a89f-639a0a34ac56 - {'path': '/', 'tag': 'd4145247-6bd4-4fe4-a89f-639a0a34ac56', 'options': 'recursive,force'} 2024-09-21T03:05:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:41 smithi018 ceph-mon[34587]: pgmap v480: 97 pgs: 97 active+clean; 49 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 561 MiB/s wr, 1.12k op/s 2024-09-21T03:05:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:05:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:05:41.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:05:41.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:41 smithi123 ceph-mon[34645]: pgmap v480: 97 pgs: 97 active+clean; 49 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 561 MiB/s wr, 1.12k op/s 2024-09-21T03:05:41.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:05:41.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:05:41.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:05:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:41 smithi016 ceph-mon[31253]: pgmap v480: 97 pgs: 97 active+clean; 49 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 561 MiB/s wr, 1.12k op/s 2024-09-21T03:05:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:05:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:05:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:05:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:43 smithi018 ceph-mon[34587]: pgmap v481: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 650 MiB/s wr, 1.63k op/s 2024-09-21T03:05:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:43 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:05:43.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:43 smithi123 ceph-mon[34645]: pgmap v481: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 650 MiB/s wr, 1.63k op/s 2024-09-21T03:05:43.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:43 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:05:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:43 smithi016 ceph-mon[31253]: pgmap v481: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 650 MiB/s wr, 1.63k op/s 2024-09-21T03:05:43.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:43 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:05:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:45 smithi018 ceph-mon[34587]: pgmap v482: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 520 MiB/s wr, 1.45k op/s 2024-09-21T03:05:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:45 smithi123 ceph-mon[34645]: pgmap v482: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 520 MiB/s wr, 1.45k op/s 2024-09-21T03:05:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:45 smithi016 ceph-mon[31253]: pgmap v482: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.5 MiB/s rd, 520 MiB/s wr, 1.45k op/s 2024-09-21T03:05:46.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:46 smithi018 ceph-mon[34587]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 MiB/s wr, 1.50k op/s 2024-09-21T03:05:46.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:46 smithi123 ceph-mon[34645]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 MiB/s wr, 1.50k op/s 2024-09-21T03:05:46.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:46 smithi016 ceph-mon[31253]: pgmap v483: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 572 MiB/s wr, 1.50k op/s 2024-09-21T03:05:48.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:48 smithi016 ceph-mon[31253]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 364 MiB/s wr, 1.04k op/s 2024-09-21T03:05:48.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:48 smithi018 ceph-mon[34587]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 364 MiB/s wr, 1.04k op/s 2024-09-21T03:05:48.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:48 smithi123 ceph-mon[34645]: pgmap v484: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 364 MiB/s wr, 1.04k op/s 2024-09-21T03:05:50.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:50 smithi016 ceph-mon[31253]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 986 op/s 2024-09-21T03:05:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:50 smithi018 ceph-mon[34587]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 986 op/s 2024-09-21T03:05:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:50 smithi123 ceph-mon[34645]: pgmap v485: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 986 op/s 2024-09-21T03:05:52.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:52 smithi016 ceph-mon[31253]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 421 MiB/s wr, 775 op/s 2024-09-21T03:05:52.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:52 smithi018 ceph-mon[34587]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 421 MiB/s wr, 775 op/s 2024-09-21T03:05:52.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:52 smithi123 ceph-mon[34645]: pgmap v486: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 421 MiB/s wr, 775 op/s 2024-09-21T03:05:54.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:54 smithi018 ceph-mon[34587]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 333 MiB/s wr, 262 op/s 2024-09-21T03:05:54.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:54 smithi123 ceph-mon[34645]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 333 MiB/s wr, 262 op/s 2024-09-21T03:05:55.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:54 smithi016 ceph-mon[31253]: pgmap v487: 97 pgs: 97 active+clean; 50 GiB data, 152 GiB used, 921 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 333 MiB/s wr, 262 op/s 2024-09-21T03:05:56.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:56 smithi018 ceph-mon[34587]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 475 MiB/s wr, 489 op/s 2024-09-21T03:05:56.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:56 smithi123 ceph-mon[34645]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 475 MiB/s wr, 489 op/s 2024-09-21T03:05:57.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:56 smithi016 ceph-mon[31253]: pgmap v488: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 475 MiB/s wr, 489 op/s 2024-09-21T03:05:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:05:58 smithi018 ceph-mon[34587]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 416 MiB/s wr, 432 op/s 2024-09-21T03:05:58.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:05:58 smithi123 ceph-mon[34645]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 416 MiB/s wr, 432 op/s 2024-09-21T03:05:59.018 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:05:58 smithi016 ceph-mon[31253]: pgmap v489: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 416 MiB/s wr, 432 op/s 2024-09-21T03:06:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:00 smithi018 ceph-mon[34587]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 451 MiB/s wr, 535 op/s 2024-09-21T03:06:00.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:00 smithi123 ceph-mon[34645]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 451 MiB/s wr, 535 op/s 2024-09-21T03:06:01.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:00 smithi016 ceph-mon[31253]: pgmap v490: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 451 MiB/s wr, 535 op/s 2024-09-21T03:06:02.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:02 smithi123 ceph-mon[34645]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 569 MiB/s wr, 694 op/s 2024-09-21T03:06:02.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:02 smithi016 ceph-mon[31253]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 569 MiB/s wr, 694 op/s 2024-09-21T03:06:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:02 smithi018 ceph-mon[34587]: pgmap v491: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 569 MiB/s wr, 694 op/s 2024-09-21T03:06:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:04 smithi018 ceph-mon[34587]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 459 MiB/s wr, 653 op/s 2024-09-21T03:06:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:04 smithi123 ceph-mon[34645]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 459 MiB/s wr, 653 op/s 2024-09-21T03:06:05.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:04 smithi016 ceph-mon[31253]: pgmap v492: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 459 MiB/s wr, 653 op/s 2024-09-21T03:06:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:06 smithi018 ceph-mon[34587]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 515 MiB/s wr, 763 op/s 2024-09-21T03:06:06.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:06 smithi123 ceph-mon[34645]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 515 MiB/s wr, 763 op/s 2024-09-21T03:06:07.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:06 smithi016 ceph-mon[31253]: pgmap v493: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 515 MiB/s wr, 763 op/s 2024-09-21T03:06:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:08 smithi018 ceph-mon[34587]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 556 op/s 2024-09-21T03:06:08.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:08 smithi123 ceph-mon[34645]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 556 op/s 2024-09-21T03:06:09.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:08 smithi016 ceph-mon[31253]: pgmap v494: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 377 MiB/s wr, 556 op/s 2024-09-21T03:06:10.759 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:06:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:10 smithi018 ceph-mon[34587]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 382 MiB/s wr, 578 op/s 2024-09-21T03:06:10.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:10 smithi016 ceph-mon[31253]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 382 MiB/s wr, 578 op/s 2024-09-21T03:06:10.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:10 smithi123 ceph-mon[34645]: pgmap v495: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 382 MiB/s wr, 578 op/s 2024-09-21T03:06:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 -- 172.21.15.16:0/578394298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c41084b0 msgr2=0x7f42c410ece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 --2- 172.21.15.16:0/578394298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c41084b0 0x7f42c410ece0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00adb0 tx=0x7f42bc02b7e0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 -- 172.21.15.16:0/578394298 shutdown_connections 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 --2- 172.21.15.16:0/578394298 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c41084b0 0x7f42c410ece0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 --2- 172.21.15.16:0/578394298 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4107b60 0x7f42c4107f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 --2- 172.21.15.16:0/578394298 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c4102270 0x7f42c4102680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.737+0000 7f42c95d9640 1 -- 172.21.15.16:0/578394298 >> 172.21.15.16:0/578394298 conn(0x7f42c4077cc0 msgr2=0x7f42c4078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.740+0000 7f42c95d9640 1 -- 172.21.15.16:0/578394298 shutdown_connections 2024-09-21T03:06:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.742+0000 7f42c95d9640 1 -- 172.21.15.16:0/578394298 wait complete. 2024-09-21T03:06:11.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.743+0000 7f42c95d9640 1 Processor -- start 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.744+0000 7f42c95d9640 1 -- start start 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.745+0000 7f42c95d9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4102270 0x7f42c410eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.745+0000 7f42c95d9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.745+0000 7f42c95d9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c41084b0 0x7f42c410c480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.746+0000 7f42c95d9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42c4111650 con 0x7f42c4107b60 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.746+0000 7f42c95d9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41114d0 con 0x7f42c4102270 2024-09-21T03:06:11.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.746+0000 7f42c95d9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41117d0 con 0x7f42c41084b0 2024-09-21T03:06:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.750+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.751+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39692/0 (socket says 172.21.15.16:39692) 2024-09-21T03:06:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.751+0000 7f42c27fc640 1 -- 172.21.15.16:0/2877642780 learned_addr learned my addr 172.21.15.16:0/2877642780 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.751+0000 7f42c37fe640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c41084b0 0x7f42c410c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.753 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.751+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4102270 0x7f42c410eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.754+0000 7f42c27fc640 1 -- 172.21.15.16:0/2877642780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c41084b0 msgr2=0x7f42c410c480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.754+0000 7f42c27fc640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c41084b0 0x7f42c410c480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.754+0000 7f42c27fc640 1 -- 172.21.15.16:0/2877642780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4102270 msgr2=0x7f42c410eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.754+0000 7f42c27fc640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4102270 0x7f42c410eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.755+0000 7f42c27fc640 1 -- 172.21.15.16:0/2877642780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42c4073e30 con 0x7f42c4107b60 2024-09-21T03:06:11.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.756+0000 7f42c27fc640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9f0 tx=0x7f42b400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.758+0000 7f42abfff640 1 -- 172.21.15.16:0/2877642780 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b400ccc0 con 0x7f42c4107b60 2024-09-21T03:06:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.758+0000 7f42abfff640 1 -- 172.21.15.16:0/2877642780 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40108e0 con 0x7f42c4107b60 2024-09-21T03:06:11.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.761+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42c4074120 con 0x7f42c4107b60 2024-09-21T03:06:11.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.763+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42c4074570 con 0x7f42c4107b60 2024-09-21T03:06:11.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.764+0000 7f42abfff640 1 -- 172.21.15.16:0/2877642780 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b4010630 con 0x7f42c4107b60 2024-09-21T03:06:11.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.780+0000 7f42abfff640 1 -- 172.21.15.16:0/2877642780 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f42b4010a90 con 0x7f42c4107b60 2024-09-21T03:06:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.781+0000 7f42abfff640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 0x7f42ac0b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.782+0000 7f42abfff640 1 -- 172.21.15.16:0/2877642780 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42b40e7a40 con 0x7f42c4107b60 2024-09-21T03:06:11.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.782+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 0x7f42ac0b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.786+0000 7f42cfb4d740 1 Processor -- start 2024-09-21T03:06:11.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.786+0000 7f42cfb4d740 1 -- start start 2024-09-21T03:06:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.787+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 0x7f42ac0b8c00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f42b80058c0 tx=0x7f42b800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.787+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.787+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab737b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.787+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab73cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582baae9e50 con 0x5582ba7a31b0 2024-09-21T03:06:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5582baad9280 con 0x5582ba7cc070 2024-09-21T03:06:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582baadd090 con 0x5582ba7ab390 2024-09-21T03:06:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42c2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42c2ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:32906/0 (socket says 172.21.15.16:32906) 2024-09-21T03:06:11.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.788+0000 7f42c2ffd640 1 -- 172.21.15.16:0/966439046 learned_addr learned my addr 172.21.15.16:0/966439046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.790+0000 7f42c27fc640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab737b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c37fe640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab73cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c2ffd640 1 -- 172.21.15.16:0/966439046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 msgr2=0x5582bab73cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab73cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c2ffd640 1 -- 172.21.15.16:0/966439046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 msgr2=0x5582bab737b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab737b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.791+0000 7f42c2ffd640 1 -- 172.21.15.16:0/966439046 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582ba837650 con 0x5582ba7cc070 2024-09-21T03:06:11.795 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.793+0000 7f42c27fc640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:06:11.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.797+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42b800ff00 tx=0x7f42b8012000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=240af40f060ee19b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.799+0000 7f42a97fa640 1 -- 172.21.15.16:0/966439046 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b8016070 con 0x5582ba7cc070 2024-09-21T03:06:11.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.799+0000 7f42a97fa640 1 -- 172.21.15.16:0/966439046 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b801b8d0 con 0x5582ba7cc070 2024-09-21T03:06:11.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.799+0000 7f42a97fa640 1 -- 172.21.15.16:0/966439046 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b801c8e0 con 0x5582ba7cc070 2024-09-21T03:06:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.802+0000 7f42cfb4d740 1 -- 172.21.15.16:0/966439046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 msgr2=0x5582bab73270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.805 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.802+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f42b800ff00 tx=0x7f42b8012000 comp rx=0 tx=0).stop 2024-09-21T03:06:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.807+0000 7f42cfb4d740 1 -- 172.21.15.16:0/966439046 shutdown_connections 2024-09-21T03:06:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.807+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab737b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.807+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab73cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.807+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/966439046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab73270 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.807+0000 7f42cfb4d740 1 -- 172.21.15.16:0/966439046 >> 172.21.15.16:0/966439046 conn(0x5582ba7c0460 msgr2=0x5582bab71160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.811+0000 7f42cfb4d740 1 -- 172.21.15.16:0/966439046 shutdown_connections 2024-09-21T03:06:11.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.811+0000 7f42cfb4d740 1 -- 172.21.15.16:0/966439046 wait complete. 2024-09-21T03:06:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.812+0000 7f42cfb4d740 1 -- start start 2024-09-21T03:06:11.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.812+0000 7f42cfb4d740 1 Processor -- start 2024-09-21T03:06:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.813+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab7e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.814+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.814+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab8ad40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.814+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5582ba93a700 con 0x5582ba7ab390 2024-09-21T03:06:11.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.814+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5582baa74110 con 0x5582ba7cc070 2024-09-21T03:06:11.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.815+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5582bab81a50 con 0x5582ba7a31b0 2024-09-21T03:06:11.817 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.815+0000 7f42cfb4d740 10 client.0 _ll_register_callbacks cb 0x5582baa8d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.816+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.816+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39710/0 (socket says 172.21.15.16:39710) 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.816+0000 7f42c27fc640 1 -- 172.21.15.16:0/431770076 learned_addr learned my addr 172.21.15.16:0/431770076 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 -- 172.21.15.16:0/431770076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 msgr2=0x5582bab7e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab7e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 -- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 msgr2=0x5582bab8ad40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab8ad40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 -- 172.21.15.16:0/431770076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582baaa1c40 con 0x5582ba7ab390 2024-09-21T03:06:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42c27fc640 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f42b40a9e20 tx=0x7f42b40b71a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7f66d44c72d8e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.817+0000 7f42a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.818+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40adf50 con 0x5582ba7ab390 2024-09-21T03:06:11.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.819+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b801b8d0 con 0x5582ba7ab390 2024-09-21T03:06:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.819+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42b40aed90 con 0x5582ba7ab390 2024-09-21T03:06:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.819+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f42b40b2330 con 0x5582ba7ab390 2024-09-21T03:06:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.820+0000 7f42cfb4d740 10 client.16026 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:11.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.820+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5582bab8d790 con 0x5582ba7ab390 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b801b8d0 con 0x5582ba7ab390 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f42b40b34c0 con 0x5582ba7ab390 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 10 client.16026 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 10 client.16026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 20 client.16026 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba78a9f0 0x5582bab91110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 4 client.16026 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.821+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582baaebda0 con 0x5582ba78a9f0 2024-09-21T03:06:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.825+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba78a9f0 0x5582bab91110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.832+0000 7f42a6ff5640 10 client.16026 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:11.834 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.832+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba78a9f0 0x5582bab91110 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec4b319f48247979 server_cookie=5371a8c9483b42f0 in_seq=0 out_seq=0 2024-09-21T03:06:11.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.836+0000 7f42a6ff5640 1 -- 172.21.15.16:0/431770076 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5582baaebda0 con 0x5582ba78a9f0 2024-09-21T03:06:11.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.836+0000 7f42a6ff5640 10 client.16026 handle_command_reply: tid=0 2024-09-21T03:06:11.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.836+0000 7f42cfb4d740 1 client.16026 shutdown 2024-09-21T03:06:11.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 msgr2=0x5582bab7ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f42b40a9e20 tx=0x7f42b40b71a0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 shutdown_connections 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba78a9f0 0x5582bab91110 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab7ef10 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab8ad40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/431770076 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab7e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 >> 172.21.15.16:0/431770076 conn(0x5582ba7c0460 msgr2=0x5582bab76450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.837+0000 7f42a6ff5640 0 client.16026 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.838+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 shutdown_connections 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.838+0000 7f42cfb4d740 1 -- 172.21.15.16:0/431770076 wait complete. 2024-09-21T03:06:11.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.838+0000 7f42cfb4d740 20 client.16026 trim_cache size 0 max 16384 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.873+0000 7f42cfb4d740 1 Processor -- start 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.873+0000 7f42cfb4d740 1 -- start start 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.873+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab76370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42c27fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39716/0 (socket says 172.21.15.16:39716) 2024-09-21T03:06:11.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42c27fc640 1 -- 172.21.15.16:0/3938701408 learned_addr learned my addr 172.21.15.16:0/3938701408 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab7bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab76370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41bc320 con 0x5582ba7ab390 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41beba0 con 0x5582ba7cc070 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41bc990 con 0x5582ba7a31b0 2024-09-21T03:06:11.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.874+0000 7f42c37fe640 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab7bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 -- 172.21.15.16:0/3938701408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 msgr2=0x5582bab76370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab76370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 -- 172.21.15.16:0/3938701408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 msgr2=0x5582bab7bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab7bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 -- 172.21.15.16:0/3938701408 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582bab71a50 con 0x5582ba7ab390 2024-09-21T03:06:11.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42c27fc640 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f42b400c6a0 tx=0x7f42b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94fd46b0ac656689 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42a8ff9640 1 -- 172.21.15.16:0/3938701408 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40a96f0 con 0x5582ba7ab390 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42a8ff9640 1 -- 172.21.15.16:0/3938701408 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40b7270 con 0x5582ba7ab390 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.875+0000 7f42a8ff9640 1 -- 172.21.15.16:0/3938701408 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42b40ac520 con 0x5582ba7ab390 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.876+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 msgr2=0x5582bab768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.876+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f42b400c6a0 tx=0x7f42b4000bc0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 shutdown_connections 2024-09-21T03:06:11.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7ab390 0x5582bab768b0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab7bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/3938701408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7a31b0 0x5582bab76370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 >> 172.21.15.16:0/3938701408 conn(0x5582ba7c0460 msgr2=0x5582bab7d250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 shutdown_connections 2024-09-21T03:06:11.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 -- 172.21.15.16:0/3938701408 wait complete. 2024-09-21T03:06:11.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 -- start start 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.877+0000 7f42cfb4d740 1 Processor -- start 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab87530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab82a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab83010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41bd060 con 0x5582ba7a31b0 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41be4d0 con 0x5582ba7cc070 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f42c41bd730 con 0x5582ba7ab390 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42cfb4d740 10 client.0 _ll_register_callbacks cb 0x7f42b40a9ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab87530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab87530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:39730/0 (socket says 172.21.15.16:39730) 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c2ffd640 1 -- 172.21.15.16:0/713363725 learned_addr learned my addr 172.21.15.16:0/713363725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab83010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c27fc640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab82a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 -- 172.21.15.16:0/713363725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 msgr2=0x5582bab82a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab82a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 -- 172.21.15.16:0/713363725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 msgr2=0x5582bab87530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab87530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.878+0000 7f42c37fe640 1 -- 172.21.15.16:0/713363725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582bab837a0 con 0x5582ba7cc070 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42c37fe640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab83010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00ad00 tx=0x7f42bc02bd80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=88f047b98a2b0021 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42a6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42bc030920 con 0x5582ba7cc070 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f42b40b7270 con 0x5582ba7cc070 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f42bc030ac0 con 0x5582ba7cc070 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f42bc00a9b0 con 0x5582ba7cc070 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42cfb4d740 10 client.25706 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:11.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.879+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5582babfbb60 con 0x5582ba7cc070 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f42b40b7270 con 0x5582ba7cc070 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f42bc045070 con 0x5582ba7cc070 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 10 client.25706 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 10 client.25706 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 20 client.25706 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:11.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba75fda0 0x5582babfe300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 4 client.25706 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582baae7850 con 0x5582ba75fda0 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.880+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba75fda0 0x5582babfe300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42c2ffd640 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba75fda0 0x5582babfe300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fe9cf7c06c2af673 server_cookie=d1464a3f76535c02 in_seq=0 out_seq=0 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42a6ff5640 10 client.25706 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42a6ff5640 1 -- 172.21.15.16:0/713363725 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+2930 (crc 0 0 0) 0x5582baae7850 con 0x5582ba75fda0 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42a6ff5640 10 client.25706 handle_command_reply: tid=0 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42cfb4d740 1 client.25706 shutdown 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 msgr2=0x5582bab83010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.881+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab83010 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f42bc00ad00 tx=0x7f42bc02bd80 comp rx=0 tx=0).stop 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 shutdown_connections 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5582ba75fda0 0x5582babfe300 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5582ba7cc070 0x5582bab83010 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42a6ff5640 0 client.25706 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:11.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5582ba7ab390 0x5582bab82a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 --2- 172.21.15.16:0/713363725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5582ba7a31b0 0x5582bab87530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 >> 172.21.15.16:0/713363725 conn(0x5582ba7c0460 msgr2=0x5582bab76370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 shutdown_connections 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 1 -- 172.21.15.16:0/713363725 wait complete. 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.882+0000 7f42cfb4d740 20 client.25706 trim_cache size 0 max 16384 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:06:11.887 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:06:11.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:11.893 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:06:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.883+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 msgr2=0x7f42ac0b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.883+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 0x7f42ac0b8c00 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f42b80058c0 tx=0x7f42b800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.883+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 msgr2=0x7f42c410be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.883+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f42b400e9f0 tx=0x7f42b400eeb0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 shutdown_connections 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f42ac0b66f0 0x7f42ac0b8c00 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f42c4107b60 0x7f42c410be50 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f42c41084b0 0x7f42c410c480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 --2- 172.21.15.16:0/2877642780 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f42c4102270 0x7f42c410eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 >> 172.21.15.16:0/2877642780 conn(0x7f42c4077cc0 msgr2=0x7f42c4105970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 shutdown_connections 2024-09-21T03:06:11.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:11.884+0000 7f42c95d9640 1 -- 172.21.15.16:0/2877642780 wait complete. 2024-09-21T03:06:12.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:12 smithi016 ceph-mon[31253]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 415 MiB/s wr, 735 op/s 2024-09-21T03:06:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:12 smithi018 ceph-mon[34587]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 415 MiB/s wr, 735 op/s 2024-09-21T03:06:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:12 smithi123 ceph-mon[34645]: pgmap v496: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 415 MiB/s wr, 735 op/s 2024-09-21T03:06:13.174 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:06:13.174 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:06:14.100 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:14.859 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:14 smithi016 ceph-mon[31253]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 811 KiB/s rd, 270 MiB/s wr, 612 op/s 2024-09-21T03:06:15.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 -- 172.21.15.16:0/2255874627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e810fb40 msgr2=0x7f54e810ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 --2- 172.21.15.16:0/2255874627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e810fb40 0x7f54e810ff50 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f54e0009ca0 tx=0x7f54e002ba30 comp rx=0 tx=0).stop 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 -- 172.21.15.16:0/2255874627 shutdown_connections 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 --2- 172.21.15.16:0/2255874627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e810fb40 0x7f54e810ff50 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 --2- 172.21.15.16:0/2255874627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e8110490 0x7f54e807b690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 --2- 172.21.15.16:0/2255874627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e810baf0 0x7f54e810bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.002+0000 7f54ef747640 1 -- 172.21.15.16:0/2255874627 >> 172.21.15.16:0/2255874627 conn(0x7f54e8070ec0 msgr2=0x7f54e80712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:15.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.003+0000 7f54ef747640 1 -- 172.21.15.16:0/2255874627 shutdown_connections 2024-09-21T03:06:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.003+0000 7f54ef747640 1 -- 172.21.15.16:0/2255874627 wait complete. 2024-09-21T03:06:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 Processor -- start 2024-09-21T03:06:15.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 -- start start 2024-09-21T03:06:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e810baf0 0x7f54e8080350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e8110490 0x7f54e8080890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:15.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54e807ecc0 con 0x7f54e8110490 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54e807eb40 con 0x7f54e810baf0 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.005+0000 7f54ef747640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e807ee40 con 0x7f54e8080dd0 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.006+0000 7f54ecce2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e8110490 0x7f54e8080890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.006+0000 7f54edce4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.006+0000 7f54edce4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37470/0 (socket says 172.21.15.16:37470) 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.006+0000 7f54edce4640 1 -- 172.21.15.16:0/2956432105 learned_addr learned my addr 172.21.15.16:0/2956432105 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.006+0000 7f54ed4e3640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e810baf0 0x7f54e8080350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.007+0000 7f54edce4640 1 -- 172.21.15.16:0/2956432105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e810baf0 msgr2=0x7f54e8080350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.007+0000 7f54edce4640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e810baf0 0x7f54e8080350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.007+0000 7f54edce4640 1 -- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e8110490 msgr2=0x7f54e8080890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.007+0000 7f54edce4640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e8110490 0x7f54e8080890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.007+0000 7f54edce4640 1 -- 172.21.15.16:0/2956432105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e81bded0 con 0x7f54e8080dd0 2024-09-21T03:06:15.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.008+0000 7f54edce4640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b3f0 tx=0x7f54d800b8b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.009+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8015020 con 0x7f54e8080dd0 2024-09-21T03:06:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.009+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8011ce0 con 0x7f54e8080dd0 2024-09-21T03:06:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.010+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8010d90 con 0x7f54e8080dd0 2024-09-21T03:06:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.010+0000 7f54ef747640 1 -- 172.21.15.16:0/2956432105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e81be1c0 con 0x7f54e8080dd0 2024-09-21T03:06:15.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.011+0000 7f54ef747640 1 -- 172.21.15.16:0/2956432105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e81be6b0 con 0x7f54e8080dd0 2024-09-21T03:06:15.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.014+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e810b5e0 con 0x7f54e8080dd0 2024-09-21T03:06:15.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.018+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f54d8018020 con 0x7f54e8080dd0 2024-09-21T03:06:15.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.020+0000 7f54de7fc640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 0x7f54d40b8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:15.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.020+0000 7f54ed4e3640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 0x7f54d40b8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:15.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.021+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f54d80e6870 con 0x7f54e8080dd0 2024-09-21T03:06:15.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.023+0000 7f54ed4e3640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 0x7f54d40b8bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005fd0 tx=0x7f54e4005e40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:15.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.026+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f54d801f020 con 0x7f54e8080dd0 2024-09-21T03:06:15.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:14 smithi018 ceph-mon[34587]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 811 KiB/s rd, 270 MiB/s wr, 612 op/s 2024-09-21T03:06:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:14 smithi123 ceph-mon[34645]: pgmap v497: 97 pgs: 97 active+clean; 50 GiB data, 153 GiB used, 919 GiB / 1.0 TiB avail; 811 KiB/s rd, 270 MiB/s wr, 612 op/s 2024-09-21T03:06:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.540+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f54e8081650 con 0x7f54e8080dd0 2024-09-21T03:06:15.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.541+0000 7f54de7fc640 1 -- 172.21.15.16:0/2956432105 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f54d80ace60 con 0x7f54e8080dd0 2024-09-21T03:06:15.543 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:06:15.543 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:06:15.543 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:06:15.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.544+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 msgr2=0x7f54d40b8bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.544+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 0x7f54d40b8bf0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f54e4005fd0 tx=0x7f54e4005e40 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.544+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 msgr2=0x7f54e81bd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.544+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f54d800b3f0 tx=0x7f54d800b8b0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 shutdown_connections 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54d40b66e0 0x7f54d40b8bf0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e8110490 0x7f54e8080890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e8080dd0 0x7f54e81bd990 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 --2- 172.21.15.16:0/2956432105 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e810baf0 0x7f54e8080350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 >> 172.21.15.16:0/2956432105 conn(0x7f54e8070ec0 msgr2=0x7f54e8073e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 shutdown_connections 2024-09-21T03:06:15.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:15.545+0000 7f54d3fff640 1 -- 172.21.15.16:0/2956432105 wait complete. 2024-09-21T03:06:15.807 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:15 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2956432105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:16.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:15 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2956432105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:16.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:06:16.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:15 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2956432105' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:16.461 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:16.885 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:16 smithi016 ceph-mon[31253]: pgmap v498: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 425 MiB/s wr, 1.71k op/s 2024-09-21T03:06:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 -- 172.21.15.16:0/1778117936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c108cf0 msgr2=0x7f870c10f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 --2- 172.21.15.16:0/1778117936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c108cf0 0x7f870c10f520 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f8704009ca0 tx=0x7f870402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 -- 172.21.15.16:0/1778117936 shutdown_connections 2024-09-21T03:06:17.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 --2- 172.21.15.16:0/1778117936 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c108cf0 0x7f870c10f520 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 --2- 172.21.15.16:0/1778117936 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c1083a0 0x7f870c1087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 --2- 172.21.15.16:0/1778117936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c102a60 0x7f870c102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 -- 172.21.15.16:0/1778117936 >> 172.21.15.16:0/1778117936 conn(0x7f870c079730 msgr2=0x7f870c079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 -- 172.21.15.16:0/1778117936 shutdown_connections 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.103+0000 7f87135fc640 1 -- 172.21.15.16:0/1778117936 wait complete. 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 Processor -- start 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 -- start start 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 0x7f870c0ffbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c108cf0 0x7f870c1000f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f870c112960 con 0x7f870c102a60 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1127e0 con 0x7f870c108cf0 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f87135fc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f870c112ae0 con 0x7f870c1083a0 2024-09-21T03:06:17.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37688/0 (socket says 172.21.15.16:37688) 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 -- 172.21.15.16:0/2954925156 learned_addr learned my addr 172.21.15.16:0/2954925156 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8710b97640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 0x7f870c0ffbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711b99640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c108cf0 0x7f870c1000f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 -- 172.21.15.16:0/2954925156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 msgr2=0x7f870c0ffbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 0x7f870c0ffbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 -- 172.21.15.16:0/2954925156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c108cf0 msgr2=0x7f870c1000f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c108cf0 0x7f870c1000f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8711398640 1 -- 172.21.15.16:0/2954925156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f870c1a8740 con 0x7f870c102a60 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.104+0000 7f8710b97640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 0x7f870c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f8711398640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9f0 tx=0x7f870800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f87027fc640 1 -- 172.21.15.16:0/2954925156 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f870800cd80 con 0x7f870c102a60 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f87027fc640 1 -- 172.21.15.16:0/2954925156 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f87080108e0 con 0x7f870c102a60 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f870c1a8a30 con 0x7f870c102a60 2024-09-21T03:06:17.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f87027fc640 1 -- 172.21.15.16:0/2954925156 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8708018980 con 0x7f870c102a60 2024-09-21T03:06:17.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.105+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f870c1a8f70 con 0x7f870c102a60 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 Processor -- start 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 -- start start 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f87027fc640 1 -- 172.21.15.16:0/2954925156 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f8708010430 con 0x7f870c102a60 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9ad1b0 0x5604ead7d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 0x5604ead7dd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604eacf3ea0 con 0x5604ea9d6070 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5604eace32d0 con 0x5604ea9ad1b0 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8719b70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604eace70e0 con 0x5604ea9b5390 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8711b99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8711b99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37696/0 (socket says 172.21.15.16:37696) 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8711b99640 1 -- 172.21.15.16:0/1223468215 learned_addr learned my addr 172.21.15.16:0/1223468215 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8710b97640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 0x5604ead7dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.108+0000 7f8711398640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9ad1b0 0x5604ead7d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 -- 172.21.15.16:0/1223468215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 msgr2=0x5604ead7dd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 0x5604ead7dd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 -- 172.21.15.16:0/1223468215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9ad1b0 msgr2=0x5604ead7d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9ad1b0 0x5604ead7d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 -- 172.21.15.16:0/1223468215 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604eaa41650 con 0x5604ea9d6070 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8710b97640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 0x5604ead7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711b99640 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8704005cc0 tx=0x7f870402f420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfcb4775b6578028 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f87027fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 0x7f86f80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f86f77fe640 1 -- 172.21.15.16:0/1223468215 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704031cf0 con 0x5604ea9d6070 2024-09-21T03:06:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f86f77fe640 1 -- 172.21.15.16:0/1223468215 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704036040 con 0x5604ea9d6070 2024-09-21T03:06:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f8711398640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 0x7f86f80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f86f77fe640 1 -- 172.21.15.16:0/1223468215 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704004040 con 0x5604ea9d6070 2024-09-21T03:06:17.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.109+0000 7f87027fc640 1 -- 172.21.15.16:0/2954925156 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f87080e7ae0 con 0x7f870c102a60 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 -- 172.21.15.16:0/1223468215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 msgr2=0x5604ead7d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8711398640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 0x7f86f80b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87080e5300 tx=0x7f87080eb630 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f8704005cc0 tx=0x7f870402f420 comp rx=0 tx=0).stop 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 -- 172.21.15.16:0/1223468215 shutdown_connections 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9b5390 0x5604ead7dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9ad1b0 0x5604ead7d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 --2- 172.21.15.16:0/1223468215 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead7d2c0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 -- 172.21.15.16:0/1223468215 >> 172.21.15.16:0/1223468215 conn(0x5604ea9ca460 msgr2=0x5604ead7b1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 -- 172.21.15.16:0/1223468215 shutdown_connections 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.110+0000 7f8719b70740 1 -- 172.21.15.16:0/1223468215 wait complete. 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 -- start start 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 Processor -- start 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead88950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9b5390 0x5604ead88e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5604eab44700 con 0x5604ea9d6070 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5604eac7e160 con 0x5604ea9b5390 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5604ead8ba40 con 0x5604ea9ad1b0 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.111+0000 7f8719b70740 10 client.0 _ll_register_callbacks cb 0x5604eac979e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37706/0 (socket says 172.21.15.16:37706) 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 -- 172.21.15.16:0/455229331 learned_addr learned my addr 172.21.15.16:0/455229331 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead88950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8710b97640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9b5390 0x5604ead88e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 -- 172.21.15.16:0/455229331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 msgr2=0x5604ead88950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead88950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 -- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9b5390 msgr2=0x5604ead88e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9b5390 0x5604ead88e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 -- 172.21.15.16:0/455229331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604eacabc90 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8711b99640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f87040094e0 tx=0x7f870403e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281a0cc2ec1025cd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f86f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8704059060 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f8704031cf0 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.112+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8704036040 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f8704031450 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 10 client.16050 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5604ead97710 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8704036040 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f8704050070 con 0x5604ea9d6070 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 10 client.16050 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 10 client.16050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 20 client.16050 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea9949f0 0x5604ead9aa70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 4 client.16050 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.113+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604eacf5df0 con 0x5604ea9949f0 2024-09-21T03:06:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.114+0000 7f8710b97640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea9949f0 0x5604ead9aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.114+0000 7f86f4ff9640 10 client.16050 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:17.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.114+0000 7f8710b97640 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea9949f0 0x5604ead9aa70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d8b65c41f60c153 server_cookie=c3937ed6028d84f6 in_seq=0 out_seq=0 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f86f4ff9640 1 -- 172.21.15.16:0/455229331 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5604eacf5df0 con 0x5604ea9949f0 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f86f4ff9640 10 client.16050 handle_command_reply: tid=0 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 client.16050 shutdown 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 msgr2=0x5604ead89410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f87040094e0 tx=0x7f870403e040 comp rx=0 tx=0).stop 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 shutdown_connections 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea9949f0 0x5604ead9aa70 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9d6070 0x5604ead89410 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9b5390 0x5604ead88e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 --2- 172.21.15.16:0/455229331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead88950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 >> 172.21.15.16:0/455229331 conn(0x5604ea9ca460 msgr2=0x5604ead80470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f86f4ff9640 0 client.16050 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:17.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 shutdown_connections 2024-09-21T03:06:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 1 -- 172.21.15.16:0/455229331 wait complete. 2024-09-21T03:06:17.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.115+0000 7f8719b70740 20 client.16050 trim_cache size 0 max 16384 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 Processor -- start 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 -- start start 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead8a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604ead7a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1af900 con 0x5604ea9b5390 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1b2180 con 0x5604ea9d6070 2024-09-21T03:06:17.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8719b70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1aff70 con 0x5604ea9ad1b0 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37720/0 (socket says 172.21.15.16:37720) 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 -- 172.21.15.16:0/1694906831 learned_addr learned my addr 172.21.15.16:0/1694906831 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8711b99640 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead8a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8711b99640 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604ead7a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 -- 172.21.15.16:0/1694906831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 msgr2=0x5604ead8a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead8a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 -- 172.21.15.16:0/1694906831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 msgr2=0x5604ead7a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604ead7a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 -- 172.21.15.16:0/1694906831 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604ead7f8d0 con 0x5604ea9b5390 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.129+0000 7f8710b97640 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86fc012730 tx=0x7f86fc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba30d9aad1b90d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.130+0000 7f86f6ffd640 1 -- 172.21.15.16:0/1694906831 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc0123d0 con 0x5604ea9b5390 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.130+0000 7f86f6ffd640 1 -- 172.21.15.16:0/1694906831 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc00f840 con 0x5604ea9b5390 2024-09-21T03:06:17.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.130+0000 7f86f6ffd640 1 -- 172.21.15.16:0/1694906831 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc003850 con 0x5604ea9b5390 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.130+0000 7f8719b70740 1 -- 172.21.15.16:0/1694906831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 msgr2=0x5604ead943d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.130+0000 7f8719b70740 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f86fc012730 tx=0x7f86fc009f90 comp rx=0 tx=0).stop 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 -- 172.21.15.16:0/1694906831 shutdown_connections 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead943d0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604ead7a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 --2- 172.21.15.16:0/1694906831 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead8a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 -- 172.21.15.16:0/1694906831 >> 172.21.15.16:0/1694906831 conn(0x5604ea9ca460 msgr2=0x5604ead93cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 -- 172.21.15.16:0/1694906831 shutdown_connections 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 -- 172.21.15.16:0/1694906831 wait complete. 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 -- start start 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 Processor -- start 2024-09-21T03:06:17.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.131+0000 7f8719b70740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead9b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604eadfc1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1b0640 con 0x5604ea9b5390 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1b1ab0 con 0x5604ea9d6070 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f870c1b0d10 con 0x5604ea9ad1b0 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8719b70740 10 client.0 _ll_register_callbacks cb 0x7f87080eb500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37726/0 (socket says 172.21.15.16:37726) 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 -- 172.21.15.16:0/1277078028 learned_addr learned my addr 172.21.15.16:0/1277078028 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8711b99640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead9b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8711b99640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604eadfc1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 -- 172.21.15.16:0/1277078028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 msgr2=0x5604ead9b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead9b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 -- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 msgr2=0x5604eadfc1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:17.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604eadfc1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 -- 172.21.15.16:0/1277078028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604eadfc810 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8711b99640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead9b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f8710b97640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86fc014010 tx=0x7f86fc012250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bd29c21f0e1a1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.132+0000 7f86f4ff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f86fc031020 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f86fc00f9f0 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f86fc00f840 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f86fc012440 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 10 client.16056 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5604eae04d30 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f86fc00f840 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f86fc010580 con 0x5604ea9b5390 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 10 client.16056 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 10 client.16056 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 20 client.16056 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:17.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea969da0 0x5604eae07510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 4 client.16056 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:06:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.133+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5604eacf18a0 con 0x5604ea969da0 2024-09-21T03:06:17.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8711b99640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea969da0 0x5604eae07510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8711b99640 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea969da0 0x5604eae07510 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5873041fec5cf293 server_cookie=8395efbd43075626 in_seq=0 out_seq=0 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f86f4ff9640 10 client.16056 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f86f4ff9640 1 -- 172.21.15.16:0/1277078028 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x7f8704036020 con 0x5604ea969da0 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f86f4ff9640 10 client.16056 handle_command_reply: tid=0 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 client.16056 shutdown 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 msgr2=0x5604ead9bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f86fc014010 tx=0x7f86fc012250 comp rx=0 tx=0).stop 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 shutdown_connections 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5604ea969da0 0x5604eae07510 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5604ea9b5390 0x5604ead9bb40 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5604ea9d6070 0x5604eadfc1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 --2- 172.21.15.16:0/1277078028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5604ea9ad1b0 0x5604ead9b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 >> 172.21.15.16:0/1277078028 conn(0x5604ea9ca460 msgr2=0x5604ead8a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.134+0000 7f86f4ff9640 0 client.16056 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 shutdown_connections 2024-09-21T03:06:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f8719b70740 1 -- 172.21.15.16:0/1277078028 wait complete. 2024-09-21T03:06:17.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f8719b70740 20 client.16056 trim_cache size 0 max 16384 2024-09-21T03:06:17.140 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 msgr2=0x7f86f80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 0x7f86f80b8b90 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f87080e5300 tx=0x7f87080eb630 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 msgr2=0x7f870c0ff670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f870800e9f0 tx=0x7f870800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 shutdown_connections 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f86f80b6680 0x7f86f80b8b90 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f870c108cf0 0x7f870c1000f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f870c1083a0 0x7f870c0ffbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.135+0000 7f87135fc640 1 --2- 172.21.15.16:0/2954925156 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f870c102a60 0x7f870c0ff670 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.136+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 >> 172.21.15.16:0/2954925156 conn(0x7f870c079730 msgr2=0x7f870c06b1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.136+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 shutdown_connections 2024-09-21T03:06:17.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:17.136+0000 7f87135fc640 1 -- 172.21.15.16:0/2954925156 wait complete. 2024-09-21T03:06:17.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:16 smithi018 ceph-mon[34587]: pgmap v498: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 425 MiB/s wr, 1.71k op/s 2024-09-21T03:06:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:16 smithi123 ceph-mon[34645]: pgmap v498: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 425 MiB/s wr, 1.71k op/s 2024-09-21T03:06:19.093 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:06:19.093 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:06:19.121 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:18 smithi016 ceph-mon[31253]: pgmap v499: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 745 KiB/s rd, 381 MiB/s wr, 1.89k op/s 2024-09-21T03:06:19.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:18 smithi018 ceph-mon[34587]: pgmap v499: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 745 KiB/s rd, 381 MiB/s wr, 1.89k op/s 2024-09-21T03:06:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:18 smithi123 ceph-mon[34645]: pgmap v499: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 745 KiB/s rd, 381 MiB/s wr, 1.89k op/s 2024-09-21T03:06:19.400 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 -- 172.21.15.16:0/134013666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b809a0d0 msgr2=0x7f74b809a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/134013666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b809a0d0 0x7f74b809a4e0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f74ac009900 tx=0x7f74ac02a410 comp rx=0 tx=0).stop 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 -- 172.21.15.16:0/134013666 shutdown_connections 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/134013666 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b809a0d0 0x7f74b809a4e0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/134013666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809aa20 0x7f74b80a1250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/134013666 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b8094790 0x7f74b8094ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 -- 172.21.15.16:0/134013666 >> 172.21.15.16:0/134013666 conn(0x7f74b800c4e0 msgr2=0x7f74b800c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.017+0000 7f74c5e8a640 1 -- 172.21.15.16:0/134013666 shutdown_connections 2024-09-21T03:06:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.018+0000 7f74c5e8a640 1 -- 172.21.15.16:0/134013666 wait complete. 2024-09-21T03:06:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.019+0000 7f74c5e8a640 1 Processor -- start 2024-09-21T03:06:20.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.019+0000 7f74c5e8a640 1 -- start start 2024-09-21T03:06:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 0x7f74b812ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 0x7f74b812f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 0x7f74b812ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 0x7f74b812ec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37740/0 (socket says 172.21.15.16:37740) 2024-09-21T03:06:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74bf7fe640 1 -- 172.21.15.16:0/3217877471 learned_addr learned my addr 172.21.15.16:0/3217877471 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:20.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b809aa20 0x7f74b8134450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b80a6040 con 0x7f74b8094790 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f74b80a5ec0 con 0x7f74b809aa20 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b80a61c0 con 0x7f74b809a0d0 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74beffd640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 0x7f74b812f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.020+0000 7f74bffff640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b809aa20 0x7f74b8134450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 -- 172.21.15.16:0/3217877471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b809aa20 msgr2=0x7f74b8134450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b809aa20 0x7f74b8134450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 -- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 msgr2=0x7f74b812ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 0x7f74b812ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b8134b80 con 0x7f74b809a0d0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74beffd640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 0x7f74b812f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f74ac02a930 tx=0x7f74ac031a20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.021+0000 7f74bcff9640 1 -- 172.21.15.16:0/3217877471 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac03f070 con 0x7f74b809a0d0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.022+0000 7f74bcff9640 1 -- 172.21.15.16:0/3217877471 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac004490 con 0x7f74b809a0d0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.022+0000 7f74bcff9640 1 -- 172.21.15.16:0/3217877471 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac035070 con 0x7f74b809a0d0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.022+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b8134e10 con 0x7f74b809a0d0 2024-09-21T03:06:20.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.023+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b8135280 con 0x7f74b809a0d0 2024-09-21T03:06:20.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.026+0000 7f74bcff9640 1 -- 172.21.15.16:0/3217877471 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f74ac03b460 con 0x7f74b809a0d0 2024-09-21T03:06:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.028+0000 7f74bcff9640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 0x7f74b00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.029+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 0x7f74b00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.029+0000 7f74bcff9640 1 -- 172.21.15.16:0/3217877471 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f74ac107b70 con 0x7f74b809a0d0 2024-09-21T03:06:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.029+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 0x7f74b00b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009c20 tx=0x7f74b4005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.030+0000 7f74cc3fe740 1 Processor -- start 2024-09-21T03:06:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.030+0000 7f74cc3fe740 1 -- start start 2024-09-21T03:06:20.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.031+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.031+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec3813b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.031+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37746/0 (socket says 172.21.15.16:37746) 2024-09-21T03:06:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec344b670 0x563ec3814090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74bf7fe640 1 -- 172.21.15.16:0/1469113005 learned_addr learned my addr 172.21.15.16:0/1469113005 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec3789e60 con 0x563ec346c350 2024-09-21T03:06:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ec3779290 con 0x563ec3443490 2024-09-21T03:06:20.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ec377d0a0 con 0x563ec344b670 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74beffd640 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec3813b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.032+0000 7f74bffff640 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec344b670 0x563ec3814090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 -- 172.21.15.16:0/1469113005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec344b670 msgr2=0x563ec3814090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec344b670 0x563ec3814090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 -- 172.21.15.16:0/1469113005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 msgr2=0x563ec3813b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec3813b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 -- 172.21.15.16:0/1469113005 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec34d7930 con 0x563ec346c350 2024-09-21T03:06:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.033+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f74b4012480 tx=0x7f74b4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b18c3c98158e0f3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.034+0000 7f74a9ffb640 1 -- 172.21.15.16:0/1469113005 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b401f620 con 0x563ec346c350 2024-09-21T03:06:20.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.034+0000 7f74a9ffb640 1 -- 172.21.15.16:0/1469113005 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74b401fc70 con 0x563ec346c350 2024-09-21T03:06:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.035+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 msgr2=0x563ec3813610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.035+0000 7f74a9ffb640 1 -- 172.21.15.16:0/1469113005 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74b401dc80 con 0x563ec346c350 2024-09-21T03:06:20.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.036+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f74b4012480 tx=0x7f74b4014000 comp rx=0 tx=0).stop 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.036+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 shutdown_connections 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.036+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec344b670 0x563ec3814090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.036+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec3813b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.036+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/1469113005 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec346c350 0x563ec3813610 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.037+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 >> 172.21.15.16:0/1469113005 conn(0x563ec3460740 msgr2=0x563ec3811500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.038 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.037+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 shutdown_connections 2024-09-21T03:06:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.037+0000 7f74cc3fe740 1 -- 172.21.15.16:0/1469113005 wait complete. 2024-09-21T03:06:20.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.038+0000 7f74cc3fe740 1 -- start start 2024-09-21T03:06:20.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.039+0000 7f74cc3fe740 1 Processor -- start 2024-09-21T03:06:20.041 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.040+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec3827980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.040+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec3827ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.040+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec3827980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.040+0000 7f74bf7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec3827980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37568/0 (socket says 172.21.15.16:37568) 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.040+0000 7f74bf7fe640 1 -- 172.21.15.16:0/4188251390 learned_addr learned my addr 172.21.15.16:0/4188251390 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec346c350 0x563ec38338f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74beffd640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec3827ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563ec35da890 con 0x563ec344b670 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563ec3713d40 con 0x563ec346c350 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74bffff640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec346c350 0x563ec38338f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.041+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563ec382a570 con 0x563ec3443490 2024-09-21T03:06:20.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.042+0000 7f74cc3fe740 10 client.0 _ll_register_callbacks cb 0x563ec3777c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74beffd640 1 -- 172.21.15.16:0/4188251390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 msgr2=0x563ec3827980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74beffd640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec3827980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74beffd640 1 -- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec346c350 msgr2=0x563ec38338f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74beffd640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec346c350 0x563ec38338f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74beffd640 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec3833e60 con 0x563ec344b670 2024-09-21T03:06:20.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.043+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74b401fc70 con 0x563ec344b670 2024-09-21T03:06:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.044+0000 7f74beffd640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec3827ec0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74ac1052e0 tx=0x7f74ac0cd8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1407d6ebcd7d8b0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.044+0000 7f749effd640 10 client.16068 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.044+0000 7f749effd640 1 -- 172.21.15.16:0/4188251390 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac048040 con 0x563ec344b670 2024-09-21T03:06:20.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.044+0000 7f749effd640 1 -- 172.21.15.16:0/4188251390 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f74ac0d2170 con 0x563ec344b670 2024-09-21T03:06:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.045+0000 7f749effd640 1 -- 172.21.15.16:0/4188251390 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f74ac0c9150 con 0x563ec344b670 2024-09-21T03:06:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.045+0000 7f74cc3fe740 10 client.16068 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.045+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=20}) -- 0x563ec3836690 con 0x563ec344b670 2024-09-21T03:06:20.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.045+0000 7f749effd640 1 -- 172.21.15.16:0/4188251390 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f74ac04b060 con 0x563ec344b670 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 10 client.16068 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 10 client.16068 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 20 client.16068 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec342acd0 0x563ec3839ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 4 client.16068 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:20.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.046+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563ec378bdb0 con 0x563ec342acd0 2024-09-21T03:06:20.052 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.051+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec342acd0 0x563ec3839ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.052+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec342acd0 0x563ec3839ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43b4612b5fde6379 server_cookie=bd3656057fbc7d5d in_seq=0 out_seq=0 2024-09-21T03:06:20.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.052+0000 7f749effd640 10 client.16068 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.053+0000 7f749effd640 1 -- 172.21.15.16:0/4188251390 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563ec378bdb0 con 0x563ec342acd0 2024-09-21T03:06:20.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.053+0000 7f749effd640 10 client.16068 handle_command_reply: tid=0 2024-09-21T03:06:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.053+0000 7f74cc3fe740 1 client.16068 shutdown 2024-09-21T03:06:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.054+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 msgr2=0x563ec3827ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.054+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec3827ec0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f74ac1052e0 tx=0x7f74ac0cd8e0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 shutdown_connections 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec342acd0 0x563ec3839ab0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f749effd640 0 client.16068 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec3827ec0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec346c350 0x563ec38338f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/4188251390 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec3827980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.055+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 >> 172.21.15.16:0/4188251390 conn(0x563ec3460740 msgr2=0x563ec380fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.056+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 shutdown_connections 2024-09-21T03:06:20.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.056+0000 7f74cc3fe740 1 -- 172.21.15.16:0/4188251390 wait complete. 2024-09-21T03:06:20.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.056+0000 7f74cc3fe740 20 client.16068 trim_cache size 0 max 16384 2024-09-21T03:06:20.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.098+0000 7f74cc3fe740 1 Processor -- start 2024-09-21T03:06:20.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 -- start start 2024-09-21T03:06:20.101 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec342acd0 0x563ec3810210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec344b670 0x563ec3824970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b813dbd0 con 0x563ec342acd0 2024-09-21T03:06:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f74b8140410 con 0x563ec344b670 2024-09-21T03:06:20.102 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.099+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b813e220 con 0x563ec3443490 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.100+0000 7f74beffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.100+0000 7f74beffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37578/0 (socket says 172.21.15.16:37578) 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.100+0000 7f74beffd640 1 -- 172.21.15.16:0/244173063 learned_addr learned my addr 172.21.15.16:0/244173063 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.100+0000 7f74bffff640 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec344b670 0x563ec3824970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.101+0000 7f74beffd640 1 -- 172.21.15.16:0/244173063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec344b670 msgr2=0x563ec3824970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.101+0000 7f74beffd640 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec344b670 0x563ec3824970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.103 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.101+0000 7f74beffd640 1 -- 172.21.15.16:0/244173063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec342acd0 msgr2=0x563ec3810210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.101+0000 7f74beffd640 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec342acd0 0x563ec3810210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.104 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.101+0000 7f74beffd640 1 -- 172.21.15.16:0/244173063 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563ec382a400 con 0x563ec3443490 2024-09-21T03:06:20.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.104+0000 7f74beffd640 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0c9fb0 tx=0x7f74ac044000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=48c53ed3f82bf686 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.104+0000 7f74a97fa640 1 -- 172.21.15.16:0/244173063 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac048040 con 0x563ec3443490 2024-09-21T03:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.105+0000 7f74a97fa640 1 -- 172.21.15.16:0/244173063 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac0ed070 con 0x563ec3443490 2024-09-21T03:06:20.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.106+0000 7f74a97fa640 1 -- 172.21.15.16:0/244173063 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0e3b90 con 0x563ec3443490 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.107+0000 7f74cc3fe740 1 -- 172.21.15.16:0/244173063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 msgr2=0x563ec38337f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.107+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f74ac0c9fb0 tx=0x7f74ac044000 comp rx=0 tx=0).stop 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 -- 172.21.15.16:0/244173063 shutdown_connections 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec344b670 0x563ec3824970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec3443490 0x563ec38337f0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/244173063 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec342acd0 0x563ec3810210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 -- 172.21.15.16:0/244173063 >> 172.21.15.16:0/244173063 conn(0x563ec3460740 msgr2=0x563ec380fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 -- 172.21.15.16:0/244173063 shutdown_connections 2024-09-21T03:06:20.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.108+0000 7f74cc3fe740 1 -- 172.21.15.16:0/244173063 wait complete. 2024-09-21T03:06:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.109+0000 7f74cc3fe740 1 -- start start 2024-09-21T03:06:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 Processor -- start 2024-09-21T03:06:20.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec342acd0 0x563ec382fd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec382d8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f74b813e8d0 con 0x563ec344b670 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f74b813fd40 con 0x563ec3443490 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.110+0000 7f74cc3fe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f74b813efa0 con 0x563ec342acd0 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.111+0000 7f74bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.111+0000 7f74bffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37772/0 (socket says 172.21.15.16:37772) 2024-09-21T03:06:20.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.111+0000 7f74bffff640 1 -- 172.21.15.16:0/394831676 learned_addr learned my addr 172.21.15.16:0/394831676 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.111+0000 7f74cc3fe740 10 client.0 _ll_register_callbacks cb 0x7f74ac0ca080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:20.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.112+0000 7f74bffff640 1 -- 172.21.15.16:0/394831676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec342acd0 msgr2=0x563ec382fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.112+0000 7f74bffff640 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec342acd0 0x563ec382fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.112+0000 7f74bffff640 1 -- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 msgr2=0x563ec382d8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.112+0000 7f74bffff640 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec382d8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.112+0000 7f74bffff640 1 -- 172.21.15.16:0/394831676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563ec382e370 con 0x563ec344b670 2024-09-21T03:06:20.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.113+0000 7f74bffff640 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74c0051d10 tx=0x7f74c0081810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a4df2debdd110a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.114+0000 7f749effd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.114+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74c006a3e0 con 0x563ec344b670 2024-09-21T03:06:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.114+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac0ed070 con 0x563ec344b670 2024-09-21T03:06:20.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.114+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f74c0074420 con 0x563ec344b670 2024-09-21T03:06:20.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.116+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f74c0074620 con 0x563ec344b670 2024-09-21T03:06:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.116+0000 7f74cc3fe740 10 client.16071 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:20.120 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.116+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563ec381e570 con 0x563ec344b670 2024-09-21T03:06:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.118+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac0ed070 con 0x563ec344b670 2024-09-21T03:06:20.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.118+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f74c008b950 con 0x563ec344b670 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 10 client.16071 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 10 client.16071 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 20 client.16071 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec346c350 0x563ec3820d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 4 client.16071 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:06:20.122 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.119+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563ec3787860 con 0x563ec346c350 2024-09-21T03:06:20.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.125+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec346c350 0x563ec3820d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.129+0000 7f74bf7fe640 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec346c350 0x563ec3820d20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=df840b502df8ec88 server_cookie=4a013c640644fa1 in_seq=0 out_seq=0 2024-09-21T03:06:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.133+0000 7f749effd640 10 client.16071 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:20.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.133+0000 7f749effd640 1 -- 172.21.15.16:0/394831676 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563ec3787860 con 0x563ec346c350 2024-09-21T03:06:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.133+0000 7f749effd640 10 client.16071 handle_command_reply: tid=0 2024-09-21T03:06:20.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.133+0000 7f74cc3fe740 1 client.16071 shutdown 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 msgr2=0x563ec382de30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f74c0051d10 tx=0x7f74c0081810 comp rx=0 tx=0).stop 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 shutdown_connections 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563ec346c350 0x563ec3820d20 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563ec344b670 0x563ec382de30 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563ec3443490 0x563ec382d8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 --2- 172.21.15.16:0/394831676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563ec342acd0 0x563ec382fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 >> 172.21.15.16:0/394831676 conn(0x563ec3460740 msgr2=0x563ec3828bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.134+0000 7f749effd640 0 client.16071 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.135+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 shutdown_connections 2024-09-21T03:06:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.135+0000 7f74cc3fe740 1 -- 172.21.15.16:0/394831676 wait complete. 2024-09-21T03:06:20.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.135+0000 7f74cc3fe740 20 client.16071 trim_cache size 0 max 16384 2024-09-21T03:06:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:06:20.140 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:06:20.140 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4c602931-953b-4cb1-8773-62796cf5e259", 2024-09-21T03:06:20.140 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:06:20.140 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:06:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.139+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 msgr2=0x7f74b00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.139+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 0x7f74b00b8c10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f74b4009c20 tx=0x7f74b4005f90 comp rx=0 tx=0).stop 2024-09-21T03:06:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.139+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 msgr2=0x7f74b812f1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:20.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.139+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 0x7f74b812f1b0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f74ac02a930 tx=0x7f74ac031a20 comp rx=0 tx=0).stop 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 shutdown_connections 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f74b00b6700 0x7f74b00b8c10 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f74b809aa20 0x7f74b8134450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f74b809a0d0 0x7f74b812f1b0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 --2- 172.21.15.16:0/3217877471 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f74b8094790 0x7f74b812ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:20.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.143+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 >> 172.21.15.16:0/3217877471 conn(0x7f74b800c4e0 msgr2=0x7f74b8096e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.144+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 shutdown_connections 2024-09-21T03:06:20.146 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:20.144+0000 7f74c5e8a640 1 -- 172.21.15.16:0/3217877471 wait complete. 2024-09-21T03:06:21.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:06:21.133 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:20 smithi016 ceph-mon[31253]: pgmap v500: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 412 KiB/s rd, 414 MiB/s wr, 2.25k op/s 2024-09-21T03:06:21.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:20 smithi018 ceph-mon[34587]: pgmap v500: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 412 KiB/s rd, 414 MiB/s wr, 2.25k op/s 2024-09-21T03:06:21.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:20 smithi123 ceph-mon[34645]: pgmap v500: 97 pgs: 97 active+clean; 51 GiB data, 157 GiB used, 915 GiB / 1.0 TiB avail; 412 KiB/s rd, 414 MiB/s wr, 2.25k op/s 2024-09-21T03:06:22.122 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:21 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:06:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:21 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:21 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:06:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:21 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:21 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:06:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:21 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:21 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:06:22.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:21 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:21 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:06:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:21 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:21 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:06:22.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:21 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:23.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:22 smithi123 ceph-mon[34645]: pgmap v501: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 829 KiB/s rd, 426 MiB/s wr, 2.55k op/s 2024-09-21T03:06:23.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:22 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:06:23.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:22 smithi016 ceph-mon[31253]: pgmap v501: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 829 KiB/s rd, 426 MiB/s wr, 2.55k op/s 2024-09-21T03:06:23.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:22 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:06:23.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:22 smithi018 ceph-mon[34587]: pgmap v501: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 829 KiB/s rd, 426 MiB/s wr, 2.55k op/s 2024-09-21T03:06:23.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:22 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:06:23.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.614+0000 7f789d7f0640 1 -- 172.21.15.16:0/1369378348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789810a9e0 msgr2=0x7f789810adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.614+0000 7f789d7f0640 1 --2- 172.21.15.16:0/1369378348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789810a9e0 0x7f789810adf0 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f788c009900 tx=0x7f788c02a440 comp rx=0 tx=0).stop 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 -- 172.21.15.16:0/1369378348 shutdown_connections 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 --2- 172.21.15.16:0/1369378348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789810a9e0 0x7f789810adf0 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 --2- 172.21.15.16:0/1369378348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789811cc30 0x7f789811f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 --2- 172.21.15.16:0/1369378348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 0x7f789810a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 -- 172.21.15.16:0/1369378348 >> 172.21.15.16:0/1369378348 conn(0x7f7898070ec0 msgr2=0x7f78980712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.618+0000 7f789d7f0640 1 -- 172.21.15.16:0/1369378348 shutdown_connections 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.619+0000 7f789d7f0640 1 -- 172.21.15.16:0/1369378348 wait complete. 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.620+0000 7f789d7f0640 1 Processor -- start 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 -- start start 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789810a9e0 0x7f78981a9410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 0x7f78981a99b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7898122960 con 0x7f789811cc30 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f78981227e0 con 0x7f7898111f40 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.624+0000 7f789d7f0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7898122ae0 con 0x7f789810a9e0 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.626+0000 7f78977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.626+0000 7f78977fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37782/0 (socket says 172.21.15.16:37782) 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.626+0000 7f78977fe640 1 -- 172.21.15.16:0/514100422 learned_addr learned my addr 172.21.15.16:0/514100422 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.627+0000 7f7896ffd640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789810a9e0 0x7f78981a9410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.627+0000 7f78967fc640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 0x7f78981a99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.628+0000 7f78977fe640 1 -- 172.21.15.16:0/514100422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789810a9e0 msgr2=0x7f78981a9410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.628+0000 7f78977fe640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789810a9e0 0x7f78981a9410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.628+0000 7f78977fe640 1 -- 172.21.15.16:0/514100422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 msgr2=0x7f78981a99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.628+0000 7f78977fe640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 0x7f78981a99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.628+0000 7f78977fe640 1 -- 172.21.15.16:0/514100422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78981af350 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.629+0000 7f78977fe640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f789000b990 tx=0x7f789000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.630+0000 7f787ffff640 1 -- 172.21.15.16:0/514100422 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7890019070 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.630+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78981af640 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.630+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7898112380 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.631+0000 7f787ffff640 1 -- 172.21.15.16:0/514100422 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f789000cba0 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.632+0000 7f787ffff640 1 -- 172.21.15.16:0/514100422 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900075d0 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.645+0000 7f78a3d64740 1 Processor -- start 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.650+0000 7f787ffff640 1 -- 172.21.15.16:0/514100422 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f789001c070 con 0x7f789811cc30 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.650+0000 7f78a3d64740 1 -- start start 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f8a070 0x5577f9331650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f787ffff640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 0x7f78800b8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 0x5577f9331b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f93320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f78a3d64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5577f92a7e50 con 0x5577f8f8a070 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.653+0000 7f78a3d64740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5577f9297280 con 0x5577f8f611b0 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f78a3d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5577f929b090 con 0x5577f8f69390 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f78967fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 0x5577f9331b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f78967fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 0x5577f9331b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:42420/0 (socket says 172.21.15.16:42420) 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f78967fc640 1 -- 172.21.15.16:0/3753496673 learned_addr learned my addr 172.21.15.16:0/3753496673 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f787ffff640 1 -- 172.21.15.16:0/514100422 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f78900e6a60 con 0x7f789811cc30 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.654+0000 7f78977fe640 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f93320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.655+0000 7f78977fe640 1 -- 172.21.15.16:0/3753496673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 msgr2=0x5577f9331b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.655+0000 7f78977fe640 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 0x5577f9331b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.655+0000 7f78977fe640 1 -- 172.21.15.16:0/3753496673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f8a070 msgr2=0x5577f9331650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.655+0000 7f78977fe640 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f8a070 0x5577f9331650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.655+0000 7f78977fe640 1 -- 172.21.15.16:0/3753496673 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f8ff5650 con 0x5577f8f69390 2024-09-21T03:06:23.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.656+0000 7f78977fe640 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f93320d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78900e41d0 tx=0x7f7890004ad0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=66a0739345875301 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.657+0000 7f787d7fa640 1 -- 172.21.15.16:0/3753496673 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900133a0 con 0x5577f8f69390 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.657+0000 7f787d7fa640 1 -- 172.21.15.16:0/3753496673 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78900ac490 con 0x5577f8f69390 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.657+0000 7f7896ffd640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 0x7f78800b8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.658+0000 7f78a3d64740 1 -- 172.21.15.16:0/3753496673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 msgr2=0x5577f93320d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f787d7fa640 1 -- 172.21.15.16:0/3753496673 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900133a0 con 0x5577f8f69390 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f93320d0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f78900e41d0 tx=0x7f7890004ad0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 -- 172.21.15.16:0/3753496673 shutdown_connections 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f93320d0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f611b0 0x5577f9331b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3753496673 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f8a070 0x5577f9331650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 -- 172.21.15.16:0/3753496673 >> 172.21.15.16:0/3753496673 conn(0x5577f8f7e460 msgr2=0x5577f932f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f7896ffd640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 0x7f78800b8c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f788800bbe0 tx=0x7f788800b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.659+0000 7f78a3d64740 1 -- 172.21.15.16:0/3753496673 shutdown_connections 2024-09-21T03:06:23.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.660+0000 7f78a3d64740 1 -- 172.21.15.16:0/3753496673 wait complete. 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.661+0000 7f78a3d64740 1 -- start start 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.661+0000 7f78a3d64740 1 Processor -- start 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f933cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f933d220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5577f90f8700 con 0x5577f8f611b0 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5577f9232110 con 0x5577f8f8a070 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5577f933fe30 con 0x5577f8f69390 2024-09-21T03:06:23.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.662+0000 7f78a3d64740 10 client.0 _ll_register_callbacks cb 0x5577f924b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.664+0000 7f78977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.664+0000 7f78977fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:42434/0 (socket says 172.21.15.16:42434) 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.664+0000 7f78977fe640 1 -- 172.21.15.16:0/4113091975 learned_addr learned my addr 172.21.15.16:0/4113091975 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.665+0000 7f7896ffd640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f933cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.667+0000 7f78977fe640 1 -- 172.21.15.16:0/4113091975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 msgr2=0x5577f933d220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.667+0000 7f78977fe640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f933d220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.667+0000 7f78977fe640 1 -- 172.21.15.16:0/4113091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 msgr2=0x5577f933cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.667+0000 7f78977fe640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f933cce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.667+0000 7f78977fe640 1 -- 172.21.15.16:0/4113091975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f925fc40 con 0x5577f8f8a070 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.669+0000 7f78977fe640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7890014cf0 tx=0x7f789000c600 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9aa9e08aa304735f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.669+0000 7f787aff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.669+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78900b1070 con 0x5577f8f8a070 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.669+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78900ac490 con 0x5577f8f8a070 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.669+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f78900abbe0 con 0x5577f8f8a070 2024-09-21T03:06:23.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.670+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f78900133a0 con 0x5577f8f8a070 2024-09-21T03:06:23.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.680+0000 7f78a3d64740 10 client.25751 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:23.682 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.680+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5577f934bb00 con 0x5577f8f8a070 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78900ac490 con 0x5577f8f8a070 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f78900ab650 con 0x5577f8f8a070 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 10 client.25751 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 10 client.25751 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 20 client.25751 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f489f0 0x5577f934efa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 4 client.25751 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:23.687 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.685+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577f92a9da0 con 0x5577f8f489f0 2024-09-21T03:06:23.690 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.688+0000 7f7896ffd640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f489f0 0x5577f934efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.694+0000 7f7896ffd640 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f489f0 0x5577f934efa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f8b01f8a055771 server_cookie=4f5e533035d6f3e2 in_seq=0 out_seq=0 2024-09-21T03:06:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.694+0000 7f787aff5640 10 client.25751 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.698+0000 7f787aff5640 1 -- 172.21.15.16:0/4113091975 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5577f92a9da0 con 0x5577f8f489f0 2024-09-21T03:06:23.699 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.698+0000 7f787aff5640 10 client.25751 handle_command_reply: tid=0 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.698+0000 7f78a3d64740 1 client.25751 shutdown 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.698+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 msgr2=0x5577f933d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.698+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7890014cf0 tx=0x7f789000c600 comp rx=0 tx=0).stop 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.699+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 shutdown_connections 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.699+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f489f0 0x5577f934efa0 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f8a070 0x5577f933d7a0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f933d220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f787aff5640 0 client.25751 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 --2- 172.21.15.16:0/4113091975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f933cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 >> 172.21.15.16:0/4113091975 conn(0x5577f8f7e460 msgr2=0x5577f9334830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 shutdown_connections 2024-09-21T03:06:23.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.700+0000 7f78a3d64740 1 -- 172.21.15.16:0/4113091975 wait complete. 2024-09-21T03:06:23.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.702+0000 7f78a3d64740 20 client.25751 trim_cache size 0 max 16384 2024-09-21T03:06:23.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.771+0000 7f78a3d64740 1 Processor -- start 2024-09-21T03:06:23.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.771+0000 7f78a3d64740 1 -- start start 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.771+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 0x5577f933ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.771+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f9332d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f69390 0x5577f932eb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78a3d64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f788400c8e0 con 0x5577f8f611b0 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78a3d64740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f788400f160 con 0x5577f8f69390 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78a3d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f788400cf50 con 0x5577f8f489f0 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f7896ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 0x5577f933ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f9332d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78967fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f9332d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:37804/0 (socket says 172.21.15.16:37804) 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.772+0000 7f78967fc640 1 -- 172.21.15.16:0/3761112382 learned_addr learned my addr 172.21.15.16:0/3761112382 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f78977fe640 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f69390 0x5577f932eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f7896ffd640 1 -- 172.21.15.16:0/3761112382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f69390 msgr2=0x5577f932eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f7896ffd640 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f69390 0x5577f932eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f7896ffd640 1 -- 172.21.15.16:0/3761112382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 msgr2=0x5577f9332d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f7896ffd640 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f9332d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.773+0000 7f7896ffd640 1 -- 172.21.15.16:0/3761112382 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577f9333940 con 0x5577f8f489f0 2024-09-21T03:06:23.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.776+0000 7f7896ffd640 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 0x5577f933ebc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5577f934f6e0 tx=0x7f7888013000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1956b7f83335b2fa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.780+0000 7f787cff9640 1 -- 172.21.15.16:0/3761112382 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78880201d0 con 0x5577f8f489f0 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.780+0000 7f787cff9640 1 -- 172.21.15.16:0/3761112382 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78880209b0 con 0x5577f8f489f0 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.780+0000 7f787cff9640 1 -- 172.21.15.16:0/3761112382 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788801e8e0 con 0x5577f8f489f0 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 -- 172.21.15.16:0/3761112382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 msgr2=0x5577f933ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 0x5577f933ebc0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x5577f934f6e0 tx=0x7f7888013000 comp rx=0 tx=0).stop 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 -- 172.21.15.16:0/3761112382 shutdown_connections 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f9332d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f69390 0x5577f932eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 --2- 172.21.15.16:0/3761112382 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f489f0 0x5577f933ebc0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.784 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.782+0000 7f78a3d64740 1 -- 172.21.15.16:0/3761112382 >> 172.21.15.16:0/3761112382 conn(0x5577f8f7e460 msgr2=0x5577f9332680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.783+0000 7f78a3d64740 1 -- 172.21.15.16:0/3761112382 shutdown_connections 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.783+0000 7f78a3d64740 1 -- 172.21.15.16:0/3761112382 wait complete. 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.784+0000 7f78a3d64740 1 -- start start 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.784+0000 7f78a3d64740 1 Processor -- start 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.785+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f489f0 0x5577f9341190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.785+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f93416d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78a3d64740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78a3d64740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f788400d620 con 0x5577f8f611b0 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78a3d64740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f788400ea90 con 0x5577f8f489f0 2024-09-21T03:06:23.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78a3d64740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f788400dcf0 con 0x5577f8f69390 2024-09-21T03:06:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78977fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78977fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:37628/0 (socket says 172.21.15.16:37628) 2024-09-21T03:06:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.786+0000 7f78977fe640 1 -- 172.21.15.16:0/780255405 learned_addr learned my addr 172.21.15.16:0/780255405 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:23.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.788+0000 7f78967fc640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f93416d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.789+0000 7f78a3d64740 10 client.0 _ll_register_callbacks cb 0x7f78900a8070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:23.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.790+0000 7f7896ffd640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f489f0 0x5577f9341190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78967fc640 1 -- 172.21.15.16:0/780255405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 msgr2=0x5577f9341c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78967fc640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78967fc640 1 -- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f489f0 msgr2=0x5577f9341190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78967fc640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f489f0 0x5577f9341190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78967fc640 1 -- 172.21.15.16:0/780255405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577f93b8aa0 con 0x5577f8f611b0 2024-09-21T03:06:23.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.794+0000 7f78977fe640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:06:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.798+0000 7f78967fc640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f93416d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f788c0092a0 tx=0x7f788c0314a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e254f168e436b4a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:23.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.799+0000 7f787aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.801+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f788c035050 con 0x5577f8f611b0 2024-09-21T03:06:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.801+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f788c004220 con 0x5577f8f611b0 2024-09-21T03:06:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.801+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78880209b0 con 0x5577f8f611b0 2024-09-21T03:06:23.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.802+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f788c004420 con 0x5577f8f611b0 2024-09-21T03:06:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.806+0000 7f78a3d64740 10 client.16086 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:23.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.806+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5577f93b9d10 con 0x5577f8f611b0 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f788c003aa0 con 0x5577f8f611b0 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f788c04c020 con 0x5577f8f611b0 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 10 client.16086 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 10 client.16086 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 20 client.16086 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f8a070 0x5577f93bc460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 4 client.16086 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:06:23.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.808+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f78880209b0 con 0x5577f8f8a070 2024-09-21T03:06:23.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.810+0000 7f7896ffd640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f8a070 0x5577f93bc460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.819+0000 7f787aff5640 10 client.16086 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:23.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.819+0000 7f7896ffd640 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f8a070 0x5577f93bc460 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35cc1824fd5aa42d server_cookie=5b90bfcd89fefe6c in_seq=0 out_seq=0 2024-09-21T03:06:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f787aff5640 1 -- 172.21.15.16:0/780255405 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3214 (crc 0 0 0) 0x7f78880209b0 con 0x5577f8f8a070 2024-09-21T03:06:23.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f787aff5640 10 client.16086 handle_command_reply: tid=0 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 client.16086 shutdown 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 msgr2=0x5577f93416d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f93416d0 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f788c0092a0 tx=0x7f788c0314a0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 shutdown_connections 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5577f8f8a070 0x5577f93bc460 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5577f8f611b0 0x5577f93416d0 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5577f8f69390 0x5577f9341c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 --2- 172.21.15.16:0/780255405 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5577f8f489f0 0x5577f9341190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 >> 172.21.15.16:0/780255405 conn(0x5577f8f7e460 msgr2=0x5577f933e920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.825+0000 7f787aff5640 0 client.16086 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.827+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 shutdown_connections 2024-09-21T03:06:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.827+0000 7f78a3d64740 1 -- 172.21.15.16:0/780255405 wait complete. 2024-09-21T03:06:23.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.828+0000 7f78a3d64740 20 client.16086 trim_cache size 0 max 16384 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.831 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.832 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.833 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.834 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:06:23.835 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:23.837 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.829+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 msgr2=0x7f78800b8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.829+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 0x7f78800b8c00 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f788800bbe0 tx=0x7f788800b390 comp rx=0 tx=0).stop 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.829+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 msgr2=0x7f78981aec50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.829+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f789000b990 tx=0x7f789000be50 comp rx=0 tx=0).stop 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 shutdown_connections 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f78800b66f0 0x7f78800b8c00 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f789811cc30 0x7f78981aec50 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7898111f40 0x7f78981a99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 --2- 172.21.15.16:0/514100422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f789810a9e0 0x7f78981a9410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 >> 172.21.15.16:0/514100422 conn(0x7f7898070ec0 msgr2=0x7f7898071e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 shutdown_connections 2024-09-21T03:06:23.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:23.830+0000 7f789d7f0640 1 -- 172.21.15.16:0/514100422 wait complete. 2024-09-21T03:06:24.963 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:06:24.963 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:06:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:24 smithi123 ceph-mon[34645]: pgmap v502: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 631 KiB/s rd, 357 MiB/s wr, 2.32k op/s 2024-09-21T03:06:25.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:24 smithi016 ceph-mon[31253]: pgmap v502: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 631 KiB/s rd, 357 MiB/s wr, 2.32k op/s 2024-09-21T03:06:25.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:24 smithi018 ceph-mon[34587]: pgmap v502: 97 pgs: 97 active+clean; 51 GiB data, 156 GiB used, 916 GiB / 1.0 TiB avail; 631 KiB/s rd, 357 MiB/s wr, 2.32k op/s 2024-09-21T03:06:25.756 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.878+0000 7f5a3b616640 1 -- 172.21.15.16:0/1577135260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34108cf0 msgr2=0x7f5a3410f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.878+0000 7f5a3b616640 1 --2- 172.21.15.16:0/1577135260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34108cf0 0x7f5a3410f520 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c009be0 tx=0x7f5a2c02a710 comp rx=0 tx=0).stop 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.879+0000 7f5a3b616640 1 -- 172.21.15.16:0/1577135260 shutdown_connections 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.879+0000 7f5a3b616640 1 --2- 172.21.15.16:0/1577135260 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a34108cf0 0x7f5a3410f520 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.879+0000 7f5a3b616640 1 --2- 172.21.15.16:0/1577135260 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a341083a0 0x7f5a341087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.880+0000 7f5a3b616640 1 --2- 172.21.15.16:0/1577135260 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a34102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:26.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.880+0000 7f5a3b616640 1 -- 172.21.15.16:0/1577135260 >> 172.21.15.16:0/1577135260 conn(0x7f5a34079730 msgr2=0x7f5a34079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.880+0000 7f5a3b616640 1 -- 172.21.15.16:0/1577135260 shutdown_connections 2024-09-21T03:06:26.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.880+0000 7f5a3b616640 1 -- 172.21.15.16:0/1577135260 wait complete. 2024-09-21T03:06:26.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 Processor -- start 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 -- start start 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a341083a0 0x7f5a3410c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 0x7f5a341ac800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111900 con 0x7f5a341083a0 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111780 con 0x7f5a34108cf0 2024-09-21T03:06:26.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.882+0000 7f5a3b616640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5a34111a80 con 0x7f5a34102a60 2024-09-21T03:06:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.884+0000 7f5a39bb3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 0x7f5a341ac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.884+0000 7f5a393b2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:26.885 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.884+0000 7f5a393b2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34178/0 (socket says 172.21.15.16:34178) 2024-09-21T03:06:26.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.884+0000 7f5a393b2640 1 -- 172.21.15.16:0/3648516725 learned_addr learned my addr 172.21.15.16:0/3648516725 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.885+0000 7f5a39bb3640 1 -- 172.21.15.16:0/3648516725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 msgr2=0x7f5a3410bb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.885+0000 7f5a39bb3640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.885+0000 7f5a39bb3640 1 -- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a341083a0 msgr2=0x7f5a3410c0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.886+0000 7f5a39bb3640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a341083a0 0x7f5a3410c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:26.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.886+0000 7f5a39bb3640 1 -- 172.21.15.16:0/3648516725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a3410c920 con 0x7f5a34108cf0 2024-09-21T03:06:26.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.886+0000 7f5a393b2640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:06:26.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.887+0000 7f5a39bb3640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 0x7f5a341ac800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02ad00 tx=0x7f5a2c003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.895+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c041020 con 0x7f5a34108cf0 2024-09-21T03:06:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.895+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a2c031700 con 0x7f5a34108cf0 2024-09-21T03:06:26.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.895+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2c036040 con 0x7f5a34108cf0 2024-09-21T03:06:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.896+0000 7f5a3b616640 1 -- 172.21.15.16:0/3648516725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a341172f0 con 0x7f5a34108cf0 2024-09-21T03:06:26.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.897+0000 7f5a3b616640 1 -- 172.21.15.16:0/3648516725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a34117750 con 0x7f5a34108cf0 2024-09-21T03:06:26.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.904+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5a18005290 con 0x7f5a34108cf0 2024-09-21T03:06:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.907+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5a2c044020 con 0x7f5a34108cf0 2024-09-21T03:06:26.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.909+0000 7f5a267fc640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 0x7f5a200b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:26.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.910+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f5a2c108090 con 0x7f5a34108cf0 2024-09-21T03:06:26.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.911+0000 7f5a393b2640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 0x7f5a200b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:26.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.914+0000 7f5a393b2640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 0x7f5a200b8d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005fd0 tx=0x7f5a2800b480 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:26.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:26.922+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f5a2c04b030 con 0x7f5a34108cf0 2024-09-21T03:06:27.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:26 smithi016 ceph-mon[31253]: pgmap v503: 97 pgs: 97 active+clean; 53 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 823 KiB/s rd, 470 MiB/s wr, 2.87k op/s 2024-09-21T03:06:27.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:26 smithi018 ceph-mon[34587]: pgmap v503: 97 pgs: 97 active+clean; 53 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 823 KiB/s rd, 470 MiB/s wr, 2.87k op/s 2024-09-21T03:06:27.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:26 smithi123 ceph-mon[34645]: pgmap v503: 97 pgs: 97 active+clean; 53 GiB data, 160 GiB used, 913 GiB / 1.0 TiB avail; 823 KiB/s rd, 470 MiB/s wr, 2.87k op/s 2024-09-21T03:06:27.474 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.472+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5a18005850 con 0x7f5a34108cf0 2024-09-21T03:06:27.476 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.475+0000 7f5a267fc640 1 -- 172.21.15.16:0/3648516725 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f5a2c0ce5a0 con 0x7f5a34108cf0 2024-09-21T03:06:27.480 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:06:27.480 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:06:27.481 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.487+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 msgr2=0x7f5a200b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.487+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 0x7f5a200b8d10 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5a28005fd0 tx=0x7f5a2800b480 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.487+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 msgr2=0x7f5a341ac800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.487+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 0x7f5a341ac800 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f5a2c02ad00 tx=0x7f5a2c003c10 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 shutdown_connections 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5a200b6800 0x7f5a200b8d10 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5a341083a0 0x7f5a3410c0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5a34108cf0 0x7f5a341ac800 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 --2- 172.21.15.16:0/3648516725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5a34102a60 0x7f5a3410bb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:27.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.488+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 >> 172.21.15.16:0/3648516725 conn(0x7f5a34079730 msgr2=0x7f5a34105200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:27.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.489+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 shutdown_connections 2024-09-21T03:06:27.495 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:27.493+0000 7f5a1ffff640 1 -- 172.21.15.16:0/3648516725 wait complete. 2024-09-21T03:06:28.316 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:28 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3648516725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:28.318 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:06:28.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:28 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3648516725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:28.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:28 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3648516725' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:06:29.103 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:29.313 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:29 smithi016 ceph-mon[31253]: pgmap v504: 97 pgs: 97 active+clean; 53 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 951 KiB/s rd, 363 MiB/s wr, 2.11k op/s 2024-09-21T03:06:29.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:29 smithi018 ceph-mon[34587]: pgmap v504: 97 pgs: 97 active+clean; 53 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 951 KiB/s rd, 363 MiB/s wr, 2.11k op/s 2024-09-21T03:06:29.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:29 smithi123 ceph-mon[34645]: pgmap v504: 97 pgs: 97 active+clean; 53 GiB data, 162 GiB used, 911 GiB / 1.0 TiB avail; 951 KiB/s rd, 363 MiB/s wr, 2.11k op/s 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 -- 172.21.15.16:0/4190932170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9410fcf0 msgr2=0x7f9f94110100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/4190932170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9410fcf0 0x7f9f94110100 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400dc90 tx=0x7f9f84029fa0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 -- 172.21.15.16:0/4190932170 shutdown_connections 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/4190932170 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9410fcf0 0x7f9f94110100 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/4190932170 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9406af90 0x7f9f94110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/4190932170 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f9410aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 -- 172.21.15.16:0/4190932170 >> 172.21.15.16:0/4190932170 conn(0x7f9f9406ee50 msgr2=0x7f9f9406f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.954+0000 7f9f9a91b640 1 -- 172.21.15.16:0/4190932170 shutdown_connections 2024-09-21T03:06:29.959 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.955+0000 7f9f9a91b640 1 -- 172.21.15.16:0/4190932170 wait complete. 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.956+0000 7f9f9a91b640 1 Processor -- start 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.956+0000 7f9f9a91b640 1 -- start start 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 0x7f9f941b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f941b1f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9410fcf0 0x7f9f941b7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94119f00 con 0x7f9f9406af90 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9f94119d80 con 0x7f9f9410fcf0 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.957+0000 7f9f9a91b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f9411a080 con 0x7f9f9410a650 2024-09-21T03:06:29.960 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.958+0000 7f9f99919640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 0x7f9f941b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9410fcf0 0x7f9f941b7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99919640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 0x7f9f941b19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48228/0 (socket says 172.21.15.16:48228) 2024-09-21T03:06:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f941b1f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:29.962 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99919640 1 -- 172.21.15.16:0/3410114865 learned_addr learned my addr 172.21.15.16:0/3410114865 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:29.963 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 -- 172.21.15.16:0/3410114865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9410fcf0 msgr2=0x7f9f941b7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9410fcf0 0x7f9f941b7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 -- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 msgr2=0x7f9f941b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 0x7f9f941b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.961+0000 7f9f99118640 1 -- 172.21.15.16:0/3410114865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9f941b7950 con 0x7f9f9410a650 2024-09-21T03:06:29.968 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.966+0000 7f9f99118640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f941b1f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400dc60 tx=0x7f9f840054f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.967+0000 7f9f8affd640 1 -- 172.21.15.16:0/3410114865 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84004440 con 0x7f9f9410a650 2024-09-21T03:06:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.967+0000 7f9f8affd640 1 -- 172.21.15.16:0/3410114865 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f84004cf0 con 0x7f9f9410a650 2024-09-21T03:06:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.967+0000 7f9f8affd640 1 -- 172.21.15.16:0/3410114865 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f84003ca0 con 0x7f9f9410a650 2024-09-21T03:06:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.967+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9f941b7be0 con 0x7f9f9410a650 2024-09-21T03:06:29.971 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.967+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9f941a53c0 con 0x7f9f9410a650 2024-09-21T03:06:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.973+0000 7f9fa20f1740 1 Processor -- start 2024-09-21T03:06:29.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.973+0000 7f9fa20f1740 1 -- start start 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.973+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.973+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa19a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d9390 0x56475caa1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56475ca17ea0 con 0x56475c6d9390 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56475ca072d0 con 0x56475c6fa070 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56475ca0b0e0 con 0x56475c6d11b0 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59988/0 (socket says 172.21.15.16:59988) 2024-09-21T03:06:29.976 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.974+0000 7f9f9a11a640 1 -- 172.21.15.16:0/3074506966 learned_addr learned my addr 172.21.15.16:0/3074506966 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.975+0000 7f9f9a11a640 1 -- 172.21.15.16:0/3074506966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 msgr2=0x56475caa19a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.975+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa19a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.975+0000 7f9f9a11a640 1 -- 172.21.15.16:0/3074506966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d9390 msgr2=0x56475caa1ee0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.975+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d9390 0x56475caa1ee0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.975+0000 7f9f9a11a640 1 -- 172.21.15.16:0/3074506966 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475c765650 con 0x56475c6fa070 2024-09-21T03:06:29.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.976+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c9c0 tx=0x7f9f8c00ce80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ff68246c1ee5cbd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.979+0000 7f9f7ffff640 1 -- 172.21.15.16:0/3074506966 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c018070 con 0x56475c6fa070 2024-09-21T03:06:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.979+0000 7f9f7ffff640 1 -- 172.21.15.16:0/3074506966 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c00ee60 con 0x56475c6fa070 2024-09-21T03:06:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.979+0000 7f9f7ffff640 1 -- 172.21.15.16:0/3074506966 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c0118b0 con 0x56475c6fa070 2024-09-21T03:06:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.984+0000 7f9fa20f1740 1 -- 172.21.15.16:0/3074506966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 msgr2=0x56475caa1460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:29.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.984+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00c9c0 tx=0x7f9f8c00ce80 comp rx=0 tx=0).stop 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.986+0000 7f9f8affd640 1 -- 172.21.15.16:0/3410114865 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9f840035f0 con 0x7f9f9410a650 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.987+0000 7f9f8affd640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 0x7f9f800b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.987+0000 7f9f8affd640 1 -- 172.21.15.16:0/3410114865 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9f84106d60 con 0x7f9f9410a650 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.989+0000 7f9fa20f1740 1 -- 172.21.15.16:0/3074506966 shutdown_connections 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.989+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d9390 0x56475caa1ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.989+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.989+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/3074506966 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa1460 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.989+0000 7f9fa20f1740 1 -- 172.21.15.16:0/3074506966 >> 172.21.15.16:0/3074506966 conn(0x56475c6ee460 msgr2=0x56475ca9f350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:29.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.990+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 0x7f9f800b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:29.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.993+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 0x7f9f800b8cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00d010 tx=0x7f9f8c013040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.997+0000 7f9fa20f1740 1 -- 172.21.15.16:0/3074506966 shutdown_connections 2024-09-21T03:06:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.997+0000 7f9fa20f1740 1 -- 172.21.15.16:0/3074506966 wait complete. 2024-09-21T03:06:30.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.998+0000 7f9fa20f1740 1 -- start start 2024-09-21T03:06:30.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:29.998+0000 7f9fa20f1740 1 Processor -- start 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caacb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6d9390 0x56475caad050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56475c868700 con 0x56475c6d11b0 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56475c9a2160 con 0x56475c6d9390 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56475caafc40 con 0x56475c6fa070 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.005+0000 7f9fa20f1740 10 client.0 _ll_register_callbacks cb 0x56475c9bb9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.006+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.006+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34220/0 (socket says 172.21.15.16:34220) 2024-09-21T03:06:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.006+0000 7f9f9a11a640 1 -- 172.21.15.16:0/2269524206 learned_addr learned my addr 172.21.15.16:0/2269524206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:30.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.007+0000 7f9f99118640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6d9390 0x56475caad050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f99919640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caacb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f9a11a640 1 -- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6d9390 msgr2=0x56475caad050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6d9390 0x56475caad050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f9a11a640 1 -- 172.21.15.16:0/2269524206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 msgr2=0x56475caacb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caacb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.008+0000 7f9f9a11a640 1 -- 172.21.15.16:0/2269524206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475c9cfc90 con 0x56475c6fa070 2024-09-21T03:06:30.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.011+0000 7f9f9a11a640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c004f90 tx=0x7f9f8c00ce80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76cddd4fafc3ab3c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:30.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.011+0000 7f9f7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.011+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f8c02e070 con 0x56475c6fa070 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.011+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9f8c02a450 con 0x56475c6fa070 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.012+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9f8c00ee60 con 0x56475c6fa070 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.012+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f8c038050 con 0x56475c6fa070 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.012+0000 7f9fa20f1740 10 client.25729 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:30.014 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.012+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56475cabb9d0 con 0x56475c6fa070 2024-09-21T03:06:30.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.013+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f8c00ee60 con 0x56475c6fa070 2024-09-21T03:06:30.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.014+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f9f8c0335d0 con 0x56475c6fa070 2024-09-21T03:06:30.016 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.014+0000 7f9fa20f1740 10 client.25729 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.014+0000 7f9fa20f1740 10 client.25729 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.014+0000 7f9fa20f1740 20 client.25729 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.014+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475c6b89f0 0x56475cabf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.015+0000 7f9fa20f1740 4 client.25729 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:30.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.015+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56475ca19df0 con 0x56475c6b89f0 2024-09-21T03:06:30.018 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.016+0000 7f9f99919640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475c6b89f0 0x56475cabf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.039+0000 7f9f99919640 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475c6b89f0 0x56475cabf350 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=553af6014eef3ef5 server_cookie=91167552ee89c35c in_seq=0 out_seq=0 2024-09-21T03:06:30.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.040+0000 7f9f7d7fa640 10 client.25729 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.047+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2269524206 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56475ca19df0 con 0x56475c6b89f0 2024-09-21T03:06:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.047+0000 7f9f7d7fa640 10 client.25729 handle_command_reply: tid=0 2024-09-21T03:06:30.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.047+0000 7f9fa20f1740 1 client.25729 shutdown 2024-09-21T03:06:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 msgr2=0x56475cab8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:30.053 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c004f90 tx=0x7f9f8c00ce80 comp rx=0 tx=0).stop 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 shutdown_connections 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475c6b89f0 0x56475cabf350 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6fa070 0x56475cab8fe0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6d9390 0x56475caad050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2269524206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caacb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 >> 172.21.15.16:0/2269524206 conn(0x56475c6ee460 msgr2=0x56475caa45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:30.054 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.052+0000 7f9f7d7fa640 0 client.25729 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:30.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.058+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 shutdown_connections 2024-09-21T03:06:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.058+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2269524206 wait complete. 2024-09-21T03:06:30.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.058+0000 7f9fa20f1740 20 client.25729 trim_cache size 0 max 16384 2024-09-21T03:06:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.095+0000 7f9fa20f1740 1 Processor -- start 2024-09-21T03:06:30.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.095+0000 7f9fa20f1740 1 -- start start 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.095+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caaa410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.096+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.096+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa2b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.096+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800c730 con 0x56475c6d11b0 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.096+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800ef00 con 0x56475c6fa070 2024-09-21T03:06:30.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.096+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800ccf0 con 0x56475c6d9390 2024-09-21T03:06:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f9a11a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34232/0 (socket says 172.21.15.16:34232) 2024-09-21T03:06:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 -- 172.21.15.16:0/1224016574 learned_addr learned my addr 172.21.15.16:0/1224016574 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:30.099 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 -- 172.21.15.16:0/1224016574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 msgr2=0x56475caa2b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa2b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 -- 172.21.15.16:0/1224016574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 msgr2=0x56475caaa410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caaa410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.097+0000 7f9f99118640 1 -- 172.21.15.16:0/1224016574 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56475cabf710 con 0x56475c6d9390 2024-09-21T03:06:30.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.098+0000 7f9f99118640 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f840c9140 tx=0x7f9f840cc2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=566d48e967e7ef3f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.157+0000 7f9f7f7fe640 1 -- 172.21.15.16:0/1224016574 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f840cc680 con 0x56475c6d9390 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.157+0000 7f9f7f7fe640 1 -- 172.21.15.16:0/1224016574 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9f840cd270 con 0x56475c6d9390 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.158+0000 7f9fa20f1740 1 -- 172.21.15.16:0/1224016574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 msgr2=0x56475cab8f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.158+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f9f840c9140 tx=0x7f9f840cc2e0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 -- 172.21.15.16:0/1224016574 shutdown_connections 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa2b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d9390 0x56475cab8f50 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/1224016574 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c6d11b0 0x56475caaa410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 -- 172.21.15.16:0/1224016574 >> 172.21.15.16:0/1224016574 conn(0x56475c6ee460 msgr2=0x56475caa2440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 -- 172.21.15.16:0/1224016574 shutdown_connections 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.159+0000 7f9fa20f1740 1 -- 172.21.15.16:0/1224016574 wait complete. 2024-09-21T03:06:30.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.160+0000 7f9fa20f1740 1 -- start start 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.160+0000 7f9fa20f1740 1 Processor -- start 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c68dda0 0x56475cb27d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800d3c0 con 0x56475c68dda0 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800e830 con 0x56475c6fa070 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9f7800da90 con 0x56475c6d11b0 2024-09-21T03:06:30.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.161+0000 7f9fa20f1740 10 client.0 _ll_register_callbacks cb 0x7f9f8c00c7c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.162+0000 7f9f99919640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.162+0000 7f9f99919640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34240/0 (socket says 172.21.15.16:34240) 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.162+0000 7f9f99919640 1 -- 172.21.15.16:0/2605493612 learned_addr learned my addr 172.21.15.16:0/2605493612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 -- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 msgr2=0x56475caa75d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa75d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 -- 172.21.15.16:0/2605493612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c68dda0 msgr2=0x56475cb27d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c68dda0 0x56475cb27d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 -- 172.21.15.16:0/2605493612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56475caa7e00 con 0x56475c6d11b0 2024-09-21T03:06:30.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.163+0000 7f9f99919640 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f90005fd0 tx=0x7f9f90013830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=844da3f9795c65e0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.165+0000 7f9f7d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:06:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.165+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9f90005220 con 0x56475c6d11b0 2024-09-21T03:06:30.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.165+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f9f900053c0 con 0x56475c6d11b0 2024-09-21T03:06:30.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.170+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56475ca158a0 con 0x56475c6d11b0 2024-09-21T03:06:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.171+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f9f900055c0 con 0x56475c6d11b0 2024-09-21T03:06:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.173+0000 7f9fa20f1740 10 client.25735 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:30.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.173+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56475cb29350 con 0x56475c6d11b0 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.174+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56475ca158a0 con 0x56475c6d11b0 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.174+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f9f90021030 con 0x56475c6d11b0 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 10 client.25735 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 10 client.25735 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 20 client.25735 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475cb29ab0 0x56475cb2bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 4 client.25735 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56475ca12cd0 con 0x56475cb29ab0 2024-09-21T03:06:30.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.175+0000 7f9f99118640 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475cb29ab0 0x56475cb2bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.184+0000 7f9f7d7fa640 10 client.25735 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.184+0000 7f9f99118640 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475cb29ab0 0x56475cb2bed0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6147854137de73d1 server_cookie=f82334563df9352b in_seq=0 out_seq=0 2024-09-21T03:06:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.188+0000 7f9f7d7fa640 1 -- 172.21.15.16:0/2605493612 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56475ca12cd0 con 0x56475cb29ab0 2024-09-21T03:06:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.188+0000 7f9f7d7fa640 10 client.25735 handle_command_reply: tid=0 2024-09-21T03:06:30.190 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.188+0000 7f9fa20f1740 1 client.25735 shutdown 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 msgr2=0x56475caa7090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f9f90005fd0 tx=0x7f9f90013830 comp rx=0 tx=0).stop 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 shutdown_connections 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56475cb29ab0 0x56475cb2bed0 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56475c68dda0 0x56475cb27d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56475c6fa070 0x56475caa75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 --2- 172.21.15.16:0/2605493612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56475c6d11b0 0x56475caa7090 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 >> 172.21.15.16:0/2605493612 conn(0x56475c6ee460 msgr2=0x56475caad050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9f7d7fa640 0 client.25735 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.189+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 shutdown_connections 2024-09-21T03:06:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.190+0000 7f9fa20f1740 1 -- 172.21.15.16:0/2605493612 wait complete. 2024-09-21T03:06:30.192 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.190+0000 7f9fa20f1740 20 client.25735 trim_cache size 0 max 16384 2024-09-21T03:06:30.194 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:06:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.192+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 msgr2=0x7f9f800b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.192+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 0x7f9f800b8cc0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f9f8c00d010 tx=0x7f9f8c013040 comp rx=0 tx=0).stop 2024-09-21T03:06:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.192+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 msgr2=0x7f9f941b1f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.192+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f941b1f80 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f9f8400dc60 tx=0x7f9f840054f0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 shutdown_connections 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9f800b67b0 0x7f9f800b8cc0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9f9410fcf0 0x7f9f941b7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9f9410a650 0x7f9f941b1f80 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 --2- 172.21.15.16:0/3410114865 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9f9406af90 0x7f9f941b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:30.198 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.196+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 >> 172.21.15.16:0/3410114865 conn(0x7f9f9406ee50 msgr2=0x7f9f9410cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.199+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 shutdown_connections 2024-09-21T03:06:30.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:30.200+0000 7f9f9a91b640 1 -- 172.21.15.16:0/3410114865 wait complete. 2024-09-21T03:06:31.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:31 smithi018 ceph-mon[34587]: pgmap v505: 97 pgs: 97 active+clean; 54 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 405 MiB/s wr, 2.14k op/s 2024-09-21T03:06:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:31 smithi123 ceph-mon[34645]: pgmap v505: 97 pgs: 97 active+clean; 54 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 405 MiB/s wr, 2.14k op/s 2024-09-21T03:06:31.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:31 smithi016 ceph-mon[31253]: pgmap v505: 97 pgs: 97 active+clean; 54 GiB data, 164 GiB used, 909 GiB / 1.0 TiB avail; 1.6 MiB/s rd, 405 MiB/s wr, 2.14k op/s 2024-09-21T03:06:32.174 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:06:32.175 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:06:32.865 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:33.282 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:33 smithi016 ceph-mon[31253]: pgmap v506: 97 pgs: 97 active+clean; 56 GiB data, 170 GiB used, 902 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 527 MiB/s wr, 2.23k op/s 2024-09-21T03:06:33.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:33 smithi018 ceph-mon[34587]: pgmap v506: 97 pgs: 97 active+clean; 56 GiB data, 170 GiB used, 902 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 527 MiB/s wr, 2.23k op/s 2024-09-21T03:06:33.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:33 smithi123 ceph-mon[34645]: pgmap v506: 97 pgs: 97 active+clean; 56 GiB data, 170 GiB used, 902 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 527 MiB/s wr, 2.23k op/s 2024-09-21T03:06:33.605 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.602+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1329108562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab41083a0 msgr2=0x7f7ab41087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.602+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1329108562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab41083a0 0x7f7ab41087b0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009860 tx=0x7f7aa402a380 comp rx=0 tx=0).stop 2024-09-21T03:06:33.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.604+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1329108562 shutdown_connections 2024-09-21T03:06:33.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.605+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1329108562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4108cf0 0x7f7ab410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.605+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1329108562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab41083a0 0x7f7ab41087b0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4009860 tx=0x7f7aa402a380 comp rx=0 tx=0).stop 2024-09-21T03:06:33.606 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.605+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1329108562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab4102a60 0x7f7ab4102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.607 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.605+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1329108562 >> 172.21.15.16:0/1329108562 conn(0x7f7ab4079730 msgr2=0x7f7ab4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.613 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.610+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1329108562 shutdown_connections 2024-09-21T03:06:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.612+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1329108562 wait complete. 2024-09-21T03:06:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.614+0000 7f7abb0f6640 1 Processor -- start 2024-09-21T03:06:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.614+0000 7f7abb0f6640 1 -- start start 2024-09-21T03:06:33.616 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.615+0000 7f7abb0f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.617 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.615+0000 7f7abb0f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab4108cf0 0x7f7ab410c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.616+0000 7f7abb0f6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab410c710 0x7f7ab4074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.618 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.616+0000 7f7abb0f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab41118c0 con 0x7f7ab4102a60 2024-09-21T03:06:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.616+0000 7f7abb0f6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111740 con 0x7f7ab410c710 2024-09-21T03:06:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.617+0000 7f7abb0f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7ab4111a40 con 0x7f7ab4108cf0 2024-09-21T03:06:33.619 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.618+0000 7f7aabfff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab4108cf0 0x7f7ab410c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.618+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.618+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48262/0 (socket says 172.21.15.16:48262) 2024-09-21T03:06:33.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.618+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1478661385 learned_addr learned my addr 172.21.15.16:0/1478661385 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:33.621 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.619+0000 7f7ab9693640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab410c710 0x7f7ab4074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.620+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1478661385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab4108cf0 msgr2=0x7f7ab410c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.620+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab4108cf0 0x7f7ab410c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.620+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1478661385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab410c710 msgr2=0x7f7ab4074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.620+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab410c710 0x7f7ab4074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.622 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.620+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1478661385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ab41b57c0 con 0x7f7ab4102a60 2024-09-21T03:06:33.623 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.621+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c920 tx=0x7f7ab000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.622+0000 7f7aa9ffb640 1 -- 172.21.15.16:0/1478661385 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab0007c70 con 0x7f7ab4102a60 2024-09-21T03:06:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.623+0000 7f7aa9ffb640 1 -- 172.21.15.16:0/1478661385 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ab0007e10 con 0x7f7ab4102a60 2024-09-21T03:06:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.623+0000 7f7aa9ffb640 1 -- 172.21.15.16:0/1478661385 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab000e7f0 con 0x7f7ab4102a60 2024-09-21T03:06:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.623+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ab41b5ab0 con 0x7f7ab4102a60 2024-09-21T03:06:33.625 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.623+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ab41b5f70 con 0x7f7ab4102a60 2024-09-21T03:06:33.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.631+0000 7f7aa9ffb640 1 -- 172.21.15.16:0/1478661385 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7ab000e990 con 0x7f7ab4102a60 2024-09-21T03:06:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.631+0000 7f7ac166a740 1 Processor -- start 2024-09-21T03:06:33.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.632+0000 7f7ac166a740 1 -- start start 2024-09-21T03:06:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fad4350 0x55981fe7b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe7b970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7ac166a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55981fdf1e60 con 0x55981fab3670 2024-09-21T03:06:33.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7aabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7aabfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48268/0 (socket says 172.21.15.16:48268) 2024-09-21T03:06:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.633+0000 7f7aabfff640 1 -- 172.21.15.16:0/775748032 learned_addr learned my addr 172.21.15.16:0/775748032 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.634+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55981fde1290 con 0x55981faab490 2024-09-21T03:06:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.634+0000 7f7ab9693640 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fad4350 0x55981fe7b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.634+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55981fde50a0 con 0x55981fad4350 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 -- 172.21.15.16:0/775748032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fad4350 msgr2=0x55981fe7b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fad4350 0x55981fe7b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 -- 172.21.15.16:0/775748032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 msgr2=0x55981fe7b970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe7b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 -- 172.21.15.16:0/775748032 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981fb3f930 con 0x55981fab3670 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aa9ffb640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 0x7f7a9c0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.635+0000 7f7aabfff640 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4035040 tx=0x7f7aa4005cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34f8b59514d075b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.636+0000 7f7ab9693640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 0x7f7a9c0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.636+0000 7f7aa9ffb640 1 -- 172.21.15.16:0/1478661385 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f7ab00e77a0 con 0x7f7ab4102a60 2024-09-21T03:06:33.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.637+0000 7f7ab9693640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 0x7f7a9c0b8cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7aac005fd0 tx=0x7f7aac00b340 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.641+0000 7f7a9affd640 1 -- 172.21.15.16:0/775748032 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4040020 con 0x55981fab3670 2024-09-21T03:06:33.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.641+0000 7f7a9affd640 1 -- 172.21.15.16:0/775748032 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa40306c0 con 0x55981fab3670 2024-09-21T03:06:33.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.641+0000 7f7a9affd640 1 -- 172.21.15.16:0/775748032 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aa4037040 con 0x55981fab3670 2024-09-21T03:06:33.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.643+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 msgr2=0x55981fe7beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.643+0000 7f7ac166a740 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f7aa4035040 tx=0x7f7aa4005cd0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.644+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 shutdown_connections 2024-09-21T03:06:33.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.644+0000 7f7ac166a740 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fab3670 0x55981fe7beb0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.644+0000 7f7ac166a740 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe7b970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.644+0000 7f7ac166a740 1 --2- 172.21.15.16:0/775748032 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fad4350 0x55981fe7b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.644+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 >> 172.21.15.16:0/775748032 conn(0x55981fac8740 msgr2=0x55981fe79320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.648+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 shutdown_connections 2024-09-21T03:06:33.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.648+0000 7f7ac166a740 1 -- 172.21.15.16:0/775748032 wait complete. 2024-09-21T03:06:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.649+0000 7f7ac166a740 1 -- start start 2024-09-21T03:06:33.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.649+0000 7f7ac166a740 1 Processor -- start 2024-09-21T03:06:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.650+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.651+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981fab3670 0x55981fe87530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.651+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe92fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.651+0000 7f7ac166a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55981fc42890 con 0x55981fad4350 2024-09-21T03:06:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.651+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.652+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:34274/0 (socket says 172.21.15.16:34274) 2024-09-21T03:06:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.652+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1849144587 learned_addr learned my addr 172.21.15.16:0/1849144587 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:33.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.652+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55981fd7bd40 con 0x55981fab3670 2024-09-21T03:06:33.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.652+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55981fe89c00 con 0x55981faab490 2024-09-21T03:06:33.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.652+0000 7f7ac166a740 10 client.0 _ll_register_callbacks cb 0x55981fddfc80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:33.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.653+0000 7f7ab9693640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe92fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.655+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981fab3670 msgr2=0x55981fe87530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.655+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981fab3670 0x55981fe87530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.655+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1849144587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 msgr2=0x55981fe92fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.655+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe92fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.655+0000 7f7ab8e92640 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55981fe93560 con 0x55981faab490 2024-09-21T03:06:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.657+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c8f0 tx=0x7f7ab00acfc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1d9fef5f67b97025 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.658+0000 7f7a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:06:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.658+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ab00b2090 con 0x55981faab490 2024-09-21T03:06:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.658+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aa40306c0 con 0x55981faab490 2024-09-21T03:06:33.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.659+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f7ab00a8f00 con 0x55981faab490 2024-09-21T03:06:33.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.659+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7ab00b6fd0 con 0x55981faab490 2024-09-21T03:06:33.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.663+0000 7f7ac166a740 10 client.25747 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:33.665 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.663+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55981fe95ef0 con 0x55981faab490 2024-09-21T03:06:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aa40306c0 con 0x55981faab490 2024-09-21T03:06:33.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f7ab00b6510 con 0x55981faab490 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7ac166a740 10 client.25747 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7ac166a740 10 client.25747 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7ac166a740 20 client.25747 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.667+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa92cd0 0x55981fe997a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.668+0000 7f7ac166a740 4 client.25747 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:33.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.668+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55981fdf3db0 con 0x55981fa92cd0 2024-09-21T03:06:33.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.668+0000 7f7aabfff640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa92cd0 0x55981fe997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.671+0000 7f7aabfff640 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa92cd0 0x55981fe997a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f7133f6ac507e581 server_cookie=e61d6fbb14b2bb3b in_seq=0 out_seq=0 2024-09-21T03:06:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.675+0000 7f7a93fff640 10 client.25747 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.675+0000 7f7a93fff640 1 -- 172.21.15.16:0/1849144587 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55981fdf3db0 con 0x55981fa92cd0 2024-09-21T03:06:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.675+0000 7f7a93fff640 10 client.25747 handle_command_reply: tid=0 2024-09-21T03:06:33.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.675+0000 7f7ac166a740 1 client.25747 shutdown 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 msgr2=0x55981fe86ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c8f0 tx=0x7f7ab00acfc0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 shutdown_connections 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa92cd0 0x55981fe997a0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe92fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981fab3670 0x55981fe87530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 --2- 172.21.15.16:0/1849144587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981faab490 0x55981fe86ff0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7a93fff640 0 client.25747 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:33.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.676+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 >> 172.21.15.16:0/1849144587 conn(0x55981fac8740 msgr2=0x55981fe7e5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.678+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 shutdown_connections 2024-09-21T03:06:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.679+0000 7f7ac166a740 1 -- 172.21.15.16:0/1849144587 wait complete. 2024-09-21T03:06:33.680 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.679+0000 7f7ac166a740 20 client.25747 trim_cache size 0 max 16384 2024-09-21T03:06:33.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.740+0000 7f7ac166a740 1 Processor -- start 2024-09-21T03:06:33.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.740+0000 7f7ac166a740 1 -- start start 2024-09-21T03:06:33.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.741+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe81550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.741+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe7cb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.741+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ac166a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000c850 con 0x55981fad4350 2024-09-21T03:06:33.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab9693640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab9693640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:48280/0 (socket says 172.21.15.16:48280) 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab9693640 1 -- 172.21.15.16:0/2136860429 learned_addr learned my addr 172.21.15.16:0/2136860429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7aabfff640 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe7cb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000f0d0 con 0x55981faab490 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000cec0 con 0x55981fab3670 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe81550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab9693640 1 -- 172.21.15.16:0/2136860429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 msgr2=0x55981fe7cb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.742+0000 7f7ab9693640 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe7cb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7ab9693640 1 -- 172.21.15.16:0/2136860429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 msgr2=0x55981fe81550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7ab9693640 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe81550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7ab9693640 1 -- 172.21.15.16:0/2136860429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55981fe85d50 con 0x55981fad4350 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7ab9693640 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7aac010780 tx=0x7f7aac012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3403a38ae5e82e3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7a9a7fc640 1 -- 172.21.15.16:0/2136860429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac010e40 con 0x55981fad4350 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7a9a7fc640 1 -- 172.21.15.16:0/2136860429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aac01ddf0 con 0x55981fad4350 2024-09-21T03:06:33.745 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.743+0000 7f7a9a7fc640 1 -- 172.21.15.16:0/2136860429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac0263d0 con 0x55981fad4350 2024-09-21T03:06:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.745+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 msgr2=0x55981fe83fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.747 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.745+0000 7f7ac166a740 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f7aac010780 tx=0x7f7aac012000 comp rx=0 tx=0).stop 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 shutdown_connections 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe83fe0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe7cb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 --2- 172.21.15.16:0/2136860429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe81550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 >> 172.21.15.16:0/2136860429 conn(0x55981fac8740 msgr2=0x55981fe7c460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 shutdown_connections 2024-09-21T03:06:33.748 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.746+0000 7f7ac166a740 1 -- 172.21.15.16:0/2136860429 wait complete. 2024-09-21T03:06:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.747+0000 7f7ac166a740 1 -- start start 2024-09-21T03:06:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.747+0000 7f7ac166a740 1 Processor -- start 2024-09-21T03:06:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe8f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe8bc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ac166a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe8c190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe8f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ab8e92640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe8f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:60074/0 (socket says 172.21.15.16:60074) 2024-09-21T03:06:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ab8e92640 1 -- 172.21.15.16:0/919505899 learned_addr learned my addr 172.21.15.16:0/919505899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:33.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7aabfff640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe8bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ab9693640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe8c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.748+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000d590 con 0x55981fad4350 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000ea00 con 0x55981faab490 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7aa000dc60 con 0x55981fab3670 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ac166a740 10 client.0 _ll_register_callbacks cb 0x7f7ab00a9a50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:33.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ab9693640 1 -- 172.21.15.16:0/919505899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 msgr2=0x55981fe8bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ab9693640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe8bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ab9693640 1 -- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 msgr2=0x55981fe8f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ab9693640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe8f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.749+0000 7f7ab9693640 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55981fe8c920 con 0x55981fad4350 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.750+0000 7f7ab9693640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe8c190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7aac010780 tx=0x7f7aac010520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6faa64f0795111 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:33.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.751+0000 7f7a93fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7aac01eda0 con 0x55981fad4350 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f7aac01d940 con 0x55981fad4350 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7aac01ddf0 con 0x55981fad4350 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7aac03e050 con 0x55981fad4350 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7ac166a740 10 client.16113 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.753+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55981ff05130 con 0x55981fad4350 2024-09-21T03:06:33.755 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7aac035070 con 0x55981fad4350 2024-09-21T03:06:33.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f7aac01e6b0 con 0x55981fad4350 2024-09-21T03:06:33.756 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 10 client.16113 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 10 client.16113 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 20 client.16113 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa68080 0x55981ff07790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 4 client.16113 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.754+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f7aac01ddf0 con 0x55981fa68080 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.755+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa68080 0x55981ff07790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:33.757 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.756+0000 7f7a93fff640 10 client.16113 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:33.759 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.757+0000 7f7ab8e92640 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa68080 0x55981ff07790 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b91658816fc2a4fa server_cookie=9684d8ccfd59f832 in_seq=0 out_seq=0 2024-09-21T03:06:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.760+0000 7f7a93fff640 1 -- 172.21.15.16:0/919505899 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f7aac01ddf0 con 0x55981fa68080 2024-09-21T03:06:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.760+0000 7f7a93fff640 10 client.16113 handle_command_reply: tid=0 2024-09-21T03:06:33.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.760+0000 7f7ac166a740 1 client.16113 shutdown 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 msgr2=0x55981fe8c190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe8c190 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7aac010780 tx=0x7f7aac010520 comp rx=0 tx=0).stop 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 shutdown_connections 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55981fa68080 0x55981ff07790 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7a93fff640 0 client.16113 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55981fad4350 0x55981fe8c190 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55981fab3670 0x55981fe8bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 --2- 172.21.15.16:0/919505899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55981faab490 0x55981fe8f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.763 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.761+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 >> 172.21.15.16:0/919505899 conn(0x55981fac8740 msgr2=0x55981fe7cdd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.762+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 shutdown_connections 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.762+0000 7f7ac166a740 1 -- 172.21.15.16:0/919505899 wait complete. 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.762+0000 7f7ac166a740 20 client.16113 trim_cache size 0 max 16384 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "4f321092-7d1c-470d-ad98-0858877c6a2e", 2024-09-21T03:06:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:06:33.765 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:06:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 msgr2=0x7f7a9c0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 0x7f7a9c0b8cc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f7aac005fd0 tx=0x7f7aac00b340 comp rx=0 tx=0).stop 2024-09-21T03:06:33.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 msgr2=0x7f7ab410bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f7ab000c920 tx=0x7f7ab000cde0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 shutdown_connections 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.763+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a9c0b67b0 0x7f7a9c0b8cc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7ab410c710 0x7f7ab4074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ab4108cf0 0x7f7ab410c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 --2- 172.21.15.16:0/1478661385 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7ab4102a60 0x7f7ab410bbc0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 >> 172.21.15.16:0/1478661385 conn(0x7f7ab4079730 msgr2=0x7f7ab4104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:33.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 shutdown_connections 2024-09-21T03:06:33.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:33.764+0000 7f7abb0f6640 1 -- 172.21.15.16:0/1478661385 wait complete. 2024-09-21T03:06:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:34 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:06:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:34 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:34.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:34 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:06:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:34 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:06:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:34 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:34.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:34 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:06:34.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:34 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:06:34.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:34 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:34.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:34 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:06:35.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:35 smithi018 ceph-mon[34587]: pgmap v507: 97 pgs: 97 active+clean; 56 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 535 MiB/s wr, 1.98k op/s 2024-09-21T03:06:35.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:35 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:35.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:35 smithi123 ceph-mon[34645]: pgmap v507: 97 pgs: 97 active+clean; 56 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 535 MiB/s wr, 1.98k op/s 2024-09-21T03:06:35.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:35 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:35.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:35 smithi016 ceph-mon[31253]: pgmap v507: 97 pgs: 97 active+clean; 56 GiB data, 172 GiB used, 901 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 535 MiB/s wr, 1.98k op/s 2024-09-21T03:06:35.513 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:35 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:06:35.514 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:06:35.876 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:06:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.926+0000 7f0a56c76640 1 -- 172.21.15.16:0/394153907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a501083d0 msgr2=0x7f0a501087e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:36.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.927+0000 7f0a56c76640 1 --2- 172.21.15.16:0/394153907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a501083d0 0x7f0a501087e0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f0a44009900 tx=0x7f0a4402a410 comp rx=0 tx=0).stop 2024-09-21T03:06:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.928+0000 7f0a56c76640 1 -- 172.21.15.16:0/394153907 shutdown_connections 2024-09-21T03:06:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.928+0000 7f0a56c76640 1 --2- 172.21.15.16:0/394153907 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a501083d0 0x7f0a501087e0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.928+0000 7f0a56c76640 1 --2- 172.21.15.16:0/394153907 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.929+0000 7f0a56c76640 1 --2- 172.21.15.16:0/394153907 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a50102ae0 0x7f0a50102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.930 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.929+0000 7f0a56c76640 1 -- 172.21.15.16:0/394153907 >> 172.21.15.16:0/394153907 conn(0x7f0a50079790 msgr2=0x7f0a50079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.938+0000 7f0a56c76640 1 -- 172.21.15.16:0/394153907 shutdown_connections 2024-09-21T03:06:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.939+0000 7f0a56c76640 1 -- 172.21.15.16:0/394153907 wait complete. 2024-09-21T03:06:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.941+0000 7f0a56c76640 1 Processor -- start 2024-09-21T03:06:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.941+0000 7f0a56c76640 1 -- start start 2024-09-21T03:06:36.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.942+0000 7f0a56c76640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a50102ae0 0x7f0a501993e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.944+0000 7f0a56c76640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 0x7f0a50199980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.944+0000 7f0a56c76640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5019ec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.945+0000 7f0a56c76640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112a90 con 0x7f0a50102ae0 2024-09-21T03:06:36.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.945+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 0x7f0a50199980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.945+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 0x7f0a50199980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59590/0 (socket says 172.21.15.16:59590) 2024-09-21T03:06:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.945+0000 7f0a4ffff640 1 -- 172.21.15.16:0/247260310 learned_addr learned my addr 172.21.15.16:0/247260310 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.946+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5019ec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.946+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112910 con 0x7f0a501083d0 2024-09-21T03:06:36.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.946+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a50112c10 con 0x7f0a50108d20 2024-09-21T03:06:36.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.948+0000 7f0a55213640 1 -- 172.21.15.16:0/247260310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 msgr2=0x7f0a50199980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:36.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.948+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 0x7f0a50199980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.948+0000 7f0a55213640 1 -- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a50102ae0 msgr2=0x7f0a501993e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.948+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a50102ae0 0x7f0a501993e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.948+0000 7f0a55213640 1 -- 172.21.15.16:0/247260310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a5019f350 con 0x7f0a50108d20 2024-09-21T03:06:36.951 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.950+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5019ec20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009fd0 tx=0x7f0a4800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:36.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.951+0000 7f0a4dffb640 1 -- 172.21.15.16:0/247260310 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4800ce20 con 0x7f0a50108d20 2024-09-21T03:06:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.951+0000 7f0a4dffb640 1 -- 172.21.15.16:0/247260310 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a48005020 con 0x7f0a50108d20 2024-09-21T03:06:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.951+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a5019f640 con 0x7f0a50108d20 2024-09-21T03:06:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.952+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a501a5da0 con 0x7f0a50108d20 2024-09-21T03:06:36.956 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.952+0000 7f0a4dffb640 1 -- 172.21.15.16:0/247260310 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a48010690 con 0x7f0a50108d20 2024-09-21T03:06:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.962+0000 7f0a5d1ea740 1 Processor -- start 2024-09-21T03:06:36.964 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.962+0000 7f0a5d1ea740 1 -- start start 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f5a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b8a1b0 0x557ef0f5ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ef0ed0e50 con 0x557ef0b8a1b0 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557ef0ec0280 con 0x557ef0b92390 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ef0ec4090 con 0x557ef0bb3070 2024-09-21T03:06:36.965 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.963+0000 7f0a4dffb640 1 -- 172.21.15.16:0/247260310 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0a48004b20 con 0x7f0a50108d20 2024-09-21T03:06:36.967 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4dffb640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 0x7f0a380b8d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f5a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59606/0 (socket says 172.21.15.16:59606) 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4ffff640 1 -- 172.21.15.16:0/2696390619 learned_addr learned my addr 172.21.15.16:0/2696390619 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.966+0000 7f0a4dffb640 1 -- 172.21.15.16:0/247260310 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f0a480e7c90 con 0x7f0a50108d20 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.967+0000 7f0a54a12640 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b8a1b0 0x557ef0f5ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.967+0000 7f0a4ffff640 1 -- 172.21.15.16:0/2696390619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 msgr2=0x557ef0f5a650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:36.969 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.967+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f5a650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.968+0000 7f0a4ffff640 1 -- 172.21.15.16:0/2696390619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b8a1b0 msgr2=0x557ef0f5ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:36.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.968+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b8a1b0 0x557ef0f5ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.968+0000 7f0a4ffff640 1 -- 172.21.15.16:0/2696390619 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef0c1e650 con 0x557ef0b92390 2024-09-21T03:06:36.970 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.968+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a44031740 tx=0x7f0a44000bc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=750b12607f9c71b2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:36.972 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.969+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 0x7f0a380b8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.975 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.973+0000 7f0a36ffd640 1 -- 172.21.15.16:0/2696390619 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44035070 con 0x557ef0b92390 2024-09-21T03:06:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.973+0000 7f0a36ffd640 1 -- 172.21.15.16:0/2696390619 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44005500 con 0x557ef0b92390 2024-09-21T03:06:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.973+0000 7f0a36ffd640 1 -- 172.21.15.16:0/2696390619 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a440049b0 con 0x557ef0b92390 2024-09-21T03:06:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/2696390619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 msgr2=0x557ef0f5b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:36.977 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a44031740 tx=0x7f0a44000bc0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/2696390619 shutdown_connections 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b8a1b0 0x557ef0f5ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0b92390 0x557ef0f5b0d0 secure :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f0a44031740 tx=0x7f0a44000bc0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/2696390619 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f5a650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/2696390619 >> 172.21.15.16:0/2696390619 conn(0x557ef0ba7460 msgr2=0x557ef0f58540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/2696390619 shutdown_connections 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.975+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/2696390619 wait complete. 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.976+0000 7f0a5d1ea740 1 -- start start 2024-09-21T03:06:36.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.976+0000 7f0a5d1ea740 1 Processor -- start 2024-09-21T03:06:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0bb3070 0x557ef0f6f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f745b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557ef0d21700 con 0x557ef0b719f0 2024-09-21T03:06:36.981 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557ef0e5b110 con 0x557ef0bb3070 2024-09-21T03:06:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.977+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557ef0f6d150 con 0x557ef0b8a1b0 2024-09-21T03:06:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.978+0000 7f0a5d1ea740 10 client.0 _ll_register_callbacks cb 0x557ef0e74990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:36.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.978+0000 7f0a55213640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 0x7f0a380b8d90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f80 tx=0x7f0a48005f70 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.982+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.982+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51674/0 (socket says 172.21.15.16:51674) 2024-09-21T03:06:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.982+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3457400570 learned_addr learned my addr 172.21.15.16:0/3457400570 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.982+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f745b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.983+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0bb3070 0x557ef0f6f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.984+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0bb3070 msgr2=0x557ef0f6f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.984+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0bb3070 0x557ef0f6f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.984+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3457400570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 msgr2=0x557ef0f745b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.984+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f745b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:36.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.984+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3457400570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef0e88c40 con 0x557ef0b8a1b0 2024-09-21T03:06:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.986+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098d0 tx=0x7f0a4402a910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fa415be8c8f09f30 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:36.987 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.986+0000 7f0a2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:06:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44035070 con 0x557ef0b8a1b0 2024-09-21T03:06:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a44005500 con 0x557ef0b8a1b0 2024-09-21T03:06:36.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f0a4403ba40 con 0x557ef0b8a1b0 2024-09-21T03:06:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a4403bc40 con 0x557ef0b8a1b0 2024-09-21T03:06:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a5d1ea740 10 client.25765 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:36.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.987+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557ef0f8e510 con 0x557ef0b8a1b0 2024-09-21T03:06:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.989+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a44005500 con 0x557ef0b8a1b0 2024-09-21T03:06:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.991+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f0a44056030 con 0x557ef0b8a1b0 2024-09-21T03:06:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 10 client.25765 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 10 client.25765 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:36.993 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 20 client.25765 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0b46da0 0x557ef0f91870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 4 client.25765 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:06:36.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.992+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557ef0ed2da0 con 0x557ef0b46da0 2024-09-21T03:06:36.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:36.996+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0b46da0 0x557ef0f91870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.008+0000 7f0a2ffff640 10 client.25765 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:37.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.008+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0b46da0 0x557ef0f91870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2eb69e1873b0efcc server_cookie=b70efe841695e71e in_seq=0 out_seq=0 2024-09-21T03:06:37.022 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3457400570 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x557ef0ed2da0 con 0x557ef0b46da0 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a2ffff640 10 client.25765 handle_command_reply: tid=0 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 client.25765 shutdown 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 msgr2=0x557ef0f6efb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098d0 tx=0x7f0a4402a910 comp rx=0 tx=0).stop 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 shutdown_connections 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0b46da0 0x557ef0f91870 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f745b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0bb3070 0x557ef0f6f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3457400570 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0b8a1b0 0x557ef0f6efb0 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0a440098d0 tx=0x7f0a4402a910 comp rx=0 tx=0).stop 2024-09-21T03:06:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 >> 172.21.15.16:0/3457400570 conn(0x557ef0ba7460 msgr2=0x557ef0f5d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.019+0000 7f0a2ffff640 0 client.25765 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.020+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 shutdown_connections 2024-09-21T03:06:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.020+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3457400570 wait complete. 2024-09-21T03:06:37.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.020+0000 7f0a5d1ea740 20 client.25765 trim_cache size 0 max 16384 2024-09-21T03:06:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.060+0000 7f0a5d1ea740 1 Processor -- start 2024-09-21T03:06:37.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.061+0000 7f0a5d1ea740 1 -- start start 2024-09-21T03:06:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.061+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f58640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.061+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0f74390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00c790 con 0x557ef0b719f0 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00ef60 con 0x557ef0f68150 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00cd50 con 0x557ef0bb3070 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a54a12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a54a12640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:51688/0 (socket says 172.21.15.16:51688) 2024-09-21T03:06:37.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.062+0000 7f0a54a12640 1 -- 172.21.15.16:0/3499797572 learned_addr learned my addr 172.21.15.16:0/3499797572 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:37.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.065+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0f74390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.066+0000 7f0a54a12640 1 -- 172.21.15.16:0/3499797572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 msgr2=0x557ef0f74390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.066+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0f74390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.066+0000 7f0a54a12640 1 -- 172.21.15.16:0/3499797572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 msgr2=0x557ef0f58640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.066+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f58640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.066+0000 7f0a54a12640 1 -- 172.21.15.16:0/3499797572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557ef0f6dd30 con 0x557ef0bb3070 2024-09-21T03:06:37.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.072+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a400093a0 tx=0x7f0a40009c30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a20ade4a6708ac7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.075+0000 7f0a367fc640 1 -- 172.21.15.16:0/3499797572 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a40020020 con 0x557ef0bb3070 2024-09-21T03:06:37.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.075+0000 7f0a367fc640 1 -- 172.21.15.16:0/3499797572 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40009e20 con 0x557ef0bb3070 2024-09-21T03:06:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a367fc640 1 -- 172.21.15.16:0/3499797572 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000f040 con 0x557ef0bb3070 2024-09-21T03:06:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3499797572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 msgr2=0x557ef0f68dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:37.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f0a400093a0 tx=0x7f0a40009c30 comp rx=0 tx=0).stop 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3499797572 shutdown_connections 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0f74390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f68dc0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3499797572 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f58640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.076+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3499797572 >> 172.21.15.16:0/3499797572 conn(0x557ef0ba7460 msgr2=0x557ef0f57f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.077+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3499797572 shutdown_connections 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.077+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3499797572 wait complete. 2024-09-21T03:06:37.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.077+0000 7f0a5d1ea740 1 -- start start 2024-09-21T03:06:37.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.078+0000 7f0a5d1ea740 1 Processor -- start 2024-09-21T03:06:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f76ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f77000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0ffafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00d420 con 0x557ef0b719f0 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00e890 con 0x557ef0f68150 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0a3c00daf0 con 0x557ef0bb3070 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.079+0000 7f0a5d1ea740 10 client.0 _ll_register_callbacks cb 0x7f0a44009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.081+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0ffafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.081+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f76ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.081+0000 7f0a4ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f76ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44382/0 (socket says 172.21.15.16:44382) 2024-09-21T03:06:37.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.081+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3740141349 learned_addr learned my addr 172.21.15.16:0/3740141349 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:06:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.081+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f77000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.082+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3740141349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 msgr2=0x557ef0f77000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.082+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f77000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.082+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3740141349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 msgr2=0x557ef0f76ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.082+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f76ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.082+0000 7f0a4ffff640 1 -- 172.21.15.16:0/3740141349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557ef0f694a0 con 0x557ef0f68150 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.083+0000 7f0a4ffff640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0ffafe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a440096c0 tx=0x7f0a4402a410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1fbb9e3e2e361bc5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.083+0000 7f0a2ffff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0a44035040 con 0x557ef0f68150 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0a40009e20 con 0x557ef0f68150 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(79..79 src has 1..79) ==== 8492+0+0 (secure 0 0 0) 0x7f0a44005020 con 0x557ef0f68150 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f0a44068050 con 0x557ef0f68150 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a5d1ea740 10 client.25787 fetch_fsmap learned FSMap version 20 2024-09-21T03:06:37.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.084+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557ef0f5f890 con 0x557ef0f68150 2024-09-21T03:06:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.085+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0a40009e20 con 0x557ef0f68150 2024-09-21T03:06:37.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.085+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f0a4404b070 con 0x557ef0f68150 2024-09-21T03:06:37.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 10 client.25787 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:06:37.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 10 client.25787 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:06:37.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 20 client.25787 populate_metadata read hostname 'smithi016' 2024-09-21T03:06:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0f5ffc0 0x557ef0f623e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:06:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 4 client.25787 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:06:37.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.087+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x557ef0ece850 con 0x557ef0f5ffc0 2024-09-21T03:06:37.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.090+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0f5ffc0 0x557ef0f623e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:06:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.093+0000 7f0a2ffff640 10 client.25787 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:37.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.093+0000 7f0a54a12640 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0f5ffc0 0x557ef0f623e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=83c201515cc3271b server_cookie=7c430ecdbbf77941 in_seq=0 out_seq=0 2024-09-21T03:06:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.095+0000 7f0a2ffff640 1 -- 172.21.15.16:0/3740141349 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3694 (crc 0 0 0) 0x557ef0ece850 con 0x557ef0f5ffc0 2024-09-21T03:06:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.095+0000 7f0a2ffff640 10 client.25787 handle_command_reply: tid=0 2024-09-21T03:06:37.097 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.095+0000 7f0a5d1ea740 1 client.25787 shutdown 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.095+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 msgr2=0x557ef0ffafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.095+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0ffafe0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f0a440096c0 tx=0x7f0a4402a410 comp rx=0 tx=0).stop 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 shutdown_connections 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557ef0f5ffc0 0x557ef0f623e0 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x557ef0f68150 0x557ef0ffafe0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x557ef0bb3070 0x557ef0f77000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 --2- 172.21.15.16:0/3740141349 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x557ef0b719f0 0x557ef0f76ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a2ffff640 0 client.25787 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:06:37.098 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.096+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 >> 172.21.15.16:0/3740141349 conn(0x557ef0ba7460 msgr2=0x557ef0f67140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.098+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 shutdown_connections 2024-09-21T03:06:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.098+0000 7f0a5d1ea740 1 -- 172.21.15.16:0/3740141349 wait complete. 2024-09-21T03:06:37.100 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.099+0000 7f0a5d1ea740 20 client.25787 trim_cache size 0 max 16384 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "4f321092-7d1c-470d-ad98-0858877c6a2e", 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:06:37.103 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:06:37.104 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:06:37.105 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.106 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.108 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:06:37.109 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.102+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 msgr2=0x7f0a380b8d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.102+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 0x7f0a380b8d90 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f0a48000f80 tx=0x7f0a48005f70 comp rx=0 tx=0).stop 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.102+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 msgr2=0x7f0a5019ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.102+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5019ec20 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0a48009fd0 tx=0x7f0a4800ef10 comp rx=0 tx=0).stop 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 shutdown_connections 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f0a380b6880 0x7f0a380b8d90 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0a50108d20 0x7f0a5019ec20 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0a501083d0 0x7f0a50199980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 --2- 172.21.15.16:0/247260310 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0a50102ae0 0x7f0a501993e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:06:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.103+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 >> 172.21.15.16:0/247260310 conn(0x7f0a50079790 msgr2=0x7f0a501063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:06:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.104+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 shutdown_connections 2024-09-21T03:06:37.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:06:37.104+0000 7f0a56c76640 1 -- 172.21.15.16:0/247260310 wait complete. 2024-09-21T03:06:37.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:37 smithi016 ceph-mon[31253]: pgmap v508: 97 pgs: 97 active+clean; 58 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 627 MiB/s wr, 2.24k op/s 2024-09-21T03:06:37.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:37 smithi018 ceph-mon[34587]: pgmap v508: 97 pgs: 97 active+clean; 58 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 627 MiB/s wr, 2.24k op/s 2024-09-21T03:06:37.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:37 smithi123 ceph-mon[34645]: pgmap v508: 97 pgs: 97 active+clean; 58 GiB data, 175 GiB used, 898 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 627 MiB/s wr, 2.24k op/s 2024-09-21T03:06:38.401 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:38 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:06:38.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:38 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:06:38.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:38 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:06:38.531 INFO:tasks.cephfs.filesystem:scrub status for tag:4f321092-7d1c-470d-ad98-0858877c6a2e - {'path': '/', 'tag': '4f321092-7d1c-470d-ad98-0858877c6a2e', 'options': 'recursive,force'} 2024-09-21T03:06:39.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:39 smithi018 ceph-mon[34587]: pgmap v509: 97 pgs: 97 active+clean; 58 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 577 MiB/s wr, 1.74k op/s 2024-09-21T03:06:39.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:39 smithi123 ceph-mon[34645]: pgmap v509: 97 pgs: 97 active+clean; 58 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 577 MiB/s wr, 1.74k op/s 2024-09-21T03:06:39.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:39 smithi016 ceph-mon[31253]: pgmap v509: 97 pgs: 97 active+clean; 58 GiB data, 177 GiB used, 896 GiB / 1.0 TiB avail; 2.9 MiB/s rd, 577 MiB/s wr, 1.74k op/s 2024-09-21T03:06:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:41 smithi016 ceph-mon[31253]: pgmap v510: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 895 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 565 MiB/s wr, 1.45k op/s 2024-09-21T03:06:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:41.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:06:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:06:41.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:06:41.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:41 smithi018 ceph-mon[34587]: pgmap v510: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 895 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 565 MiB/s wr, 1.45k op/s 2024-09-21T03:06:41.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:41.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:06:41.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:06:41.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:06:41.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:41 smithi123 ceph-mon[34645]: pgmap v510: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 895 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 565 MiB/s wr, 1.45k op/s 2024-09-21T03:06:41.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:06:41.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:06:41.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:06:41.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:06:42.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:42 smithi018 ceph-mon[34587]: pgmap v511: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 577 MiB/s wr, 1.42k op/s 2024-09-21T03:06:42.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:42 smithi123 ceph-mon[34645]: pgmap v511: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 577 MiB/s wr, 1.42k op/s 2024-09-21T03:06:42.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:42 smithi016 ceph-mon[31253]: pgmap v511: 97 pgs: 97 active+clean; 58 GiB data, 178 GiB used, 894 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 577 MiB/s wr, 1.42k op/s 2024-09-21T03:06:44.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:44 smithi018 ceph-mon[34587]: pgmap v512: 97 pgs: 97 active+clean; 59 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 437 MiB/s wr, 1.07k op/s 2024-09-21T03:06:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:44 smithi123 ceph-mon[34645]: pgmap v512: 97 pgs: 97 active+clean; 59 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 437 MiB/s wr, 1.07k op/s 2024-09-21T03:06:45.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:44 smithi016 ceph-mon[31253]: pgmap v512: 97 pgs: 97 active+clean; 59 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.2 MiB/s rd, 437 MiB/s wr, 1.07k op/s 2024-09-21T03:06:46.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:46 smithi018 ceph-mon[34587]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 469 MiB/s wr, 1.16k op/s 2024-09-21T03:06:46.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:46 smithi123 ceph-mon[34645]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 469 MiB/s wr, 1.16k op/s 2024-09-21T03:06:47.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:46 smithi016 ceph-mon[31253]: pgmap v513: 97 pgs: 97 active+clean; 59 GiB data, 180 GiB used, 893 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 469 MiB/s wr, 1.16k op/s 2024-09-21T03:06:48.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:48 smithi018 ceph-mon[34587]: pgmap v514: 97 pgs: 97 active+clean; 60 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 434 MiB/s wr, 913 op/s 2024-09-21T03:06:48.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:48 smithi123 ceph-mon[34645]: pgmap v514: 97 pgs: 97 active+clean; 60 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 434 MiB/s wr, 913 op/s 2024-09-21T03:06:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:48 smithi016 ceph-mon[31253]: pgmap v514: 97 pgs: 97 active+clean; 60 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 434 MiB/s wr, 913 op/s 2024-09-21T03:06:50.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:50 smithi018 ceph-mon[34587]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 424 MiB/s wr, 857 op/s 2024-09-21T03:06:50.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:50 smithi123 ceph-mon[34645]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 424 MiB/s wr, 857 op/s 2024-09-21T03:06:51.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:50 smithi016 ceph-mon[31253]: pgmap v515: 97 pgs: 97 active+clean; 60 GiB data, 182 GiB used, 891 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 424 MiB/s wr, 857 op/s 2024-09-21T03:06:52.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:52 smithi018 ceph-mon[34587]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 516 MiB/s wr, 896 op/s 2024-09-21T03:06:52.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:52 smithi123 ceph-mon[34645]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 516 MiB/s wr, 896 op/s 2024-09-21T03:06:53.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:52 smithi016 ceph-mon[31253]: pgmap v516: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 516 MiB/s wr, 896 op/s 2024-09-21T03:06:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:54 smithi018 ceph-mon[34587]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 466 MiB/s wr, 637 op/s 2024-09-21T03:06:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:54 smithi123 ceph-mon[34645]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 466 MiB/s wr, 637 op/s 2024-09-21T03:06:55.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:54 smithi016 ceph-mon[31253]: pgmap v517: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 890 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 466 MiB/s wr, 637 op/s 2024-09-21T03:06:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:56 smithi018 ceph-mon[34587]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 889 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 524 MiB/s wr, 809 op/s 2024-09-21T03:06:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:56 smithi123 ceph-mon[34645]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 889 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 524 MiB/s wr, 809 op/s 2024-09-21T03:06:57.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:56 smithi016 ceph-mon[31253]: pgmap v518: 97 pgs: 97 active+clean; 60 GiB data, 183 GiB used, 889 GiB / 1.0 TiB avail; 2.1 MiB/s rd, 524 MiB/s wr, 809 op/s 2024-09-21T03:06:59.068 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:06:58 smithi016 ceph-mon[31253]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 501 MiB/s wr, 853 op/s 2024-09-21T03:06:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:06:58 smithi018 ceph-mon[34587]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 501 MiB/s wr, 853 op/s 2024-09-21T03:06:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:06:58 smithi123 ceph-mon[34645]: pgmap v519: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 501 MiB/s wr, 853 op/s 2024-09-21T03:07:01.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:00 smithi016 ceph-mon[31253]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 468 MiB/s wr, 927 op/s 2024-09-21T03:07:01.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:00 smithi018 ceph-mon[34587]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 468 MiB/s wr, 927 op/s 2024-09-21T03:07:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:00 smithi123 ceph-mon[34645]: pgmap v520: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 468 MiB/s wr, 927 op/s 2024-09-21T03:07:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:02 smithi018 ceph-mon[34587]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 497 MiB/s wr, 988 op/s 2024-09-21T03:07:03.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:02 smithi123 ceph-mon[34645]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 497 MiB/s wr, 988 op/s 2024-09-21T03:07:03.269 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:02 smithi016 ceph-mon[31253]: pgmap v521: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 497 MiB/s wr, 988 op/s 2024-09-21T03:07:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:04 smithi018 ceph-mon[34587]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 387 MiB/s wr, 901 op/s 2024-09-21T03:07:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:04 smithi123 ceph-mon[34645]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 387 MiB/s wr, 901 op/s 2024-09-21T03:07:05.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:04 smithi016 ceph-mon[31253]: pgmap v522: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 387 MiB/s wr, 901 op/s 2024-09-21T03:07:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:06 smithi123 ceph-mon[34645]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 414 MiB/s wr, 921 op/s 2024-09-21T03:07:07.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:06 smithi016 ceph-mon[31253]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 414 MiB/s wr, 921 op/s 2024-09-21T03:07:07.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:06 smithi018 ceph-mon[34587]: pgmap v523: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 414 MiB/s wr, 921 op/s 2024-09-21T03:07:08.532 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:07:08.905 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:09.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:08 smithi123 ceph-mon[34645]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 656 op/s 2024-09-21T03:07:09.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:08 smithi123 ceph-mon[34645]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:07:09.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:08 smithi018 ceph-mon[34587]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 656 op/s 2024-09-21T03:07:09.234 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:08 smithi018 ceph-mon[34587]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:07:09.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:08 smithi016 ceph-mon[31253]: pgmap v524: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 366 MiB/s wr, 656 op/s 2024-09-21T03:07:09.268 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:08 smithi016 ceph-mon[31253]: osdmap e80: 12 total, 12 up, 12 in 2024-09-21T03:07:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/1485178132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 msgr2=0x7f7fe01098c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/1485178132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe01098c0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f7fd4009860 tx=0x7f7fd402a340 comp rx=0 tx=0).stop 2024-09-21T03:07:09.626 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/1485178132 shutdown_connections 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/1485178132 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0109e00 0x7f7fe0110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/1485178132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe01098c0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/1485178132 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/1485178132 >> 172.21.15.16:0/1485178132 conn(0x7f7fe0077c60 msgr2=0x7f7fe00780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/1485178132 shutdown_connections 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.624+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/1485178132 wait complete. 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 Processor -- start 2024-09-21T03:07:09.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 -- start start 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe0113ac0 con 0x7f7fe0101f70 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe0113940 con 0x7f7fe0109e00 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.625+0000 7f7fe5d2d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe0113c40 con 0x7f7fe01094b0 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:58360/0 (socket says 172.21.15.16:58360) 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58476/0 (socket says 172.21.15.16:58476) 2024-09-21T03:07:09.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 -- 172.21.15.16:0/263997959 learned_addr learned my addr 172.21.15.16:0/263997959 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 -- 172.21.15.16:0/263997959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 msgr2=0x7f7fe0103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe0103450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 -- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 msgr2=0x7f7fe0102f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 -- 172.21.15.16:0/263997959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fe01a8770 con 0x7f7fe0109e00 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.626+0000 7f7fdffff640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800efc0 tx=0x7f7fd800c800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.627+0000 7f7fdcff9640 1 -- 172.21.15.16:0/263997959 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd8003970 con 0x7f7fe0109e00 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.627+0000 7f7fdcff9640 1 -- 172.21.15.16:0/263997959 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd8004020 con 0x7f7fe0109e00 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.627+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fe01a8a60 con 0x7f7fe0109e00 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.627+0000 7f7fdcff9640 1 -- 172.21.15.16:0/263997959 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd800b940 con 0x7f7fe0109e00 2024-09-21T03:07:09.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.627+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fe01a8f20 con 0x7f7fe0109e00 2024-09-21T03:07:09.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.631+0000 7f7fec2a1740 1 Processor -- start 2024-09-21T03:07:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.632+0000 7f7fdcff9640 1 -- 172.21.15.16:0/263997959 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7fd801d020 con 0x7f7fe0109e00 2024-09-21T03:07:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.632+0000 7f7fec2a1740 1 -- start start 2024-09-21T03:07:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.634 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a731b0 0x565314e437b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a7b390 0x565314e43cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565314db9e50 con 0x565314a9c070 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565314da9280 con 0x565314a7b390 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565314dad090 con 0x565314a731b0 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58484/0 (socket says 172.21.15.16:58484) 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fdeffd640 1 -- 172.21.15.16:0/205599632 learned_addr learned my addr 172.21.15.16:0/205599632 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:09.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.633+0000 7f7fdcff9640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 0x7f7fc80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 -- 172.21.15.16:0/205599632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a731b0 msgr2=0x565314e437b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a731b0 0x565314e437b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 -- 172.21.15.16:0/205599632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a7b390 msgr2=0x565314e43cf0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:09.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a7b390 0x565314e43cf0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.637 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 -- 172.21.15.16:0/205599632 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565314b07650 con 0x565314a9c070 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdcff9640 1 -- 172.21.15.16:0/263997959 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f7fd80e7650 con 0x7f7fe0109e00 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7fd402a930 tx=0x7f7fd4009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51a87d0d52b368c3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.634+0000 7f7fdffff640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 0x7f7fc80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.635+0000 7f7fc5ffb640 1 -- 172.21.15.16:0/205599632 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4037040 con 0x565314a9c070 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.635+0000 7f7fc5ffb640 1 -- 172.21.15.16:0/205599632 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd4030e10 con 0x565314a9c070 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.635+0000 7f7fc5ffb640 1 -- 172.21.15.16:0/205599632 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4003850 con 0x565314a9c070 2024-09-21T03:07:09.638 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fdffff640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 0x7f7fc80b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80ad3f0 tx=0x7f7fd800ceb0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 -- 172.21.15.16:0/205599632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 msgr2=0x565314e43270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f7fd402a930 tx=0x7f7fd4009540 comp rx=0 tx=0).stop 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 -- 172.21.15.16:0/205599632 shutdown_connections 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a7b390 0x565314e43cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a731b0 0x565314e437b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/205599632 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a9c070 0x565314e43270 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.636+0000 7f7fec2a1740 1 -- 172.21.15.16:0/205599632 >> 172.21.15.16:0/205599632 conn(0x565314a90460 msgr2=0x565314e41160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.637+0000 7f7fec2a1740 1 -- 172.21.15.16:0/205599632 shutdown_connections 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.637+0000 7f7fec2a1740 1 -- 172.21.15.16:0/205599632 wait complete. 2024-09-21T03:07:09.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.637+0000 7f7fec2a1740 1 -- start start 2024-09-21T03:07:09.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.637+0000 7f7fec2a1740 1 Processor -- start 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e4e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e5ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x565314c0a700 con 0x565314a7b390 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x565314d44110 con 0x565314a731b0 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x565314e519f0 con 0x565314a9c070 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fec2a1740 10 client.0 _ll_register_callbacks cb 0x565314d5d990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e4e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.638+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58494/0 (socket says 172.21.15.16:58494) 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/3051212180 learned_addr learned my addr 172.21.15.16:0/3051212180 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e5ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/3051212180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 msgr2=0x565314e5ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e5ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 msgr2=0x565314e4e9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e4e9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.639+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/3051212180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565314d71c40 con 0x565314a7b390 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.640+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00116c0 tx=0x7f7fd0011b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6fe37847be8a94a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.640+0000 7f7fbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.641+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd000fd30 con 0x565314a7b390 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.641+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f7fd0004020 con 0x565314a7b390 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.641+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd4030e10 con 0x565314a7b390 2024-09-21T03:07:09.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.642+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0004220 con 0x565314a7b390 2024-09-21T03:07:09.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.642+0000 7f7fec2a1740 10 client.16137 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:09.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.642+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565314e5d7f0 con 0x565314a7b390 2024-09-21T03:07:09.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.643+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd00043c0 con 0x565314a7b390 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.643+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f7fd0004c20 con 0x565314a7b390 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.643+0000 7f7fec2a1740 10 client.16137 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fec2a1740 10 client.16137 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fec2a1740 20 client.16137 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a5a9f0 0x565314e60d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fec2a1740 4 client.16137 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f7fd4030e10 con 0x565314a5a9f0 2024-09-21T03:07:09.646 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.644+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a5a9f0 0x565314e60d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.645+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a5a9f0 0x565314e60d50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c715f5fbc95172c server_cookie=c7bbbf7f4f46b513 in_seq=0 out_seq=0 2024-09-21T03:07:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.645+0000 7f7fbeffd640 10 client.16137 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.646+0000 7f7fbeffd640 1 -- 172.21.15.16:0/3051212180 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f7fd4030e10 con 0x565314a5a9f0 2024-09-21T03:07:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.646+0000 7f7fbeffd640 10 client.16137 handle_command_reply: tid=0 2024-09-21T03:07:09.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.646+0000 7f7fec2a1740 1 client.16137 shutdown 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 msgr2=0x565314e4ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f7fd00116c0 tx=0x7f7fd0011b80 comp rx=0 tx=0).stop 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 shutdown_connections 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a5a9f0 0x565314e60d50 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e4ef10 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e5ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/3051212180 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e4e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 >> 172.21.15.16:0/3051212180 conn(0x565314a90460 msgr2=0x565314e46450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.648+0000 7f7fbeffd640 0 client.16137 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:09.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.649+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 shutdown_connections 2024-09-21T03:07:09.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.649+0000 7f7fec2a1740 1 -- 172.21.15.16:0/3051212180 wait complete. 2024-09-21T03:07:09.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.650+0000 7f7fec2a1740 20 client.16137 trim_cache size 0 max 16384 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 Processor -- start 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 -- start start 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a7b390 0x565314e4bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a9c070 0x565314e4c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01af9b0 con 0x565314a731b0 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b2210 con 0x565314a9c070 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b0000 con 0x565314a7b390 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.684+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58498/0 (socket says 172.21.15.16:58498) 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 -- 172.21.15.16:0/616426286 learned_addr learned my addr 172.21.15.16:0/616426286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a7b390 0x565314e4bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 -- 172.21.15.16:0/616426286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a7b390 msgr2=0x565314e4bbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a7b390 0x565314e4bbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 -- 172.21.15.16:0/616426286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a9c070 msgr2=0x565314e4c100 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a9c070 0x565314e4c100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 -- 172.21.15.16:0/616426286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x565314e4fc90 con 0x565314a731b0 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40303f0 tx=0x7f7fd4034040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7242ab5b24f6d6c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.685+0000 7f7fc57fa640 1 -- 172.21.15.16:0/616426286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4037040 con 0x565314a731b0 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.686+0000 7f7fc57fa640 1 -- 172.21.15.16:0/616426286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd4030e10 con 0x565314a731b0 2024-09-21T03:07:09.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.686+0000 7f7fc57fa640 1 -- 172.21.15.16:0/616426286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40054c0 con 0x565314a731b0 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- 172.21.15.16:0/616426286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 msgr2=0x565314e449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40303f0 tx=0x7f7fd4034040 comp rx=0 tx=0).stop 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- 172.21.15.16:0/616426286 shutdown_connections 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a9c070 0x565314e4c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a7b390 0x565314e4bbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/616426286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a731b0 0x565314e449b0 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- 172.21.15.16:0/616426286 >> 172.21.15.16:0/616426286 conn(0x565314a90460 msgr2=0x565314e442a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- 172.21.15.16:0/616426286 shutdown_connections 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- 172.21.15.16:0/616426286 wait complete. 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 -- start start 2024-09-21T03:07:09.691 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.687+0000 7f7fec2a1740 1 Processor -- start 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e575d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e55190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b06d0 con 0x565314a7b390 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b1b40 con 0x565314a731b0 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7fe01b0da0 con 0x565314a9c070 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fec2a1740 10 client.0 _ll_register_callbacks cb 0x7f7fd0009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fdf7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58502/0 (socket says 172.21.15.16:58502) 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/1330685052 learned_addr learned my addr 172.21.15.16:0/1330685052 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e55190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.688+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e575d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/1330685052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 msgr2=0x565314e55190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e55190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.692 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 msgr2=0x565314e575d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e575d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 -- 172.21.15.16:0/1330685052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565314eca720 con 0x565314a7b390 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fdf7fe640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0006010 tx=0x7f7fd000f810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=324db853cfcafb2e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fbeffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fd0032060 con 0x565314a7b390 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fd4030e10 con 0x565314a7b390 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.689+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f7fd0003c30 con 0x565314a7b390 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f7fd0027070 con 0x565314a7b390 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 10 client.16143 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:09.693 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x565314ecb7f0 con 0x565314a7b390 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fd4030e10 con 0x565314a7b390 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f7fd0004880 con 0x565314a7b390 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 10 client.16143 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 10 client.16143 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 20 client.16143 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:09.695 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.690+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a2fda0 0x565314ecdea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.691+0000 7f7fec2a1740 4 client.16143 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.691+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x565314dbbda0 con 0x565314a2fda0 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.691+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a2fda0 0x565314ecdea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.692+0000 7f7fdeffd640 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a2fda0 0x565314ecdea0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bd28a6227ad7ab3 server_cookie=311972a9b7089728 in_seq=0 out_seq=0 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.692+0000 7f7fbeffd640 10 client.16143 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.692+0000 7f7fbeffd640 1 -- 172.21.15.16:0/1330685052 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3498 (crc 0 0 0) 0x565314dbbda0 con 0x565314a2fda0 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.692+0000 7f7fbeffd640 10 client.16143 handle_command_reply: tid=0 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.692+0000 7f7fec2a1740 1 client.16143 shutdown 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 msgr2=0x565314e54aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f7fd0006010 tx=0x7f7fd000f810 comp rx=0 tx=0).stop 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 shutdown_connections 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x565314a2fda0 0x565314ecdea0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x565314a7b390 0x565314e54aa0 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x565314a9c070 0x565314e55190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 --2- 172.21.15.16:0/1330685052 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x565314a731b0 0x565314e575d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 >> 172.21.15.16:0/1330685052 conn(0x565314a90460 msgr2=0x565314e4fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fbeffd640 0 client.16143 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 shutdown_connections 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 1 -- 172.21.15.16:0/1330685052 wait complete. 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.693+0000 7f7fec2a1740 20 client.16143 trim_cache size 0 max 16384 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:07:09.697 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.698 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.699 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.700 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.701 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:07:09.702 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:09.703 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:07:09.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 msgr2=0x7f7fc80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 0x7f7fc80b8b90 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f7fd80ad3f0 tx=0x7f7fd800ceb0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 msgr2=0x7f7fe0103990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f7fd800efc0 tx=0x7f7fd800c800 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 shutdown_connections 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7fc80b6680 0x7f7fc80b8b90 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7fe0109e00 0x7f7fe0103990 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe01094b0 0x7f7fe0103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 --2- 172.21.15.16:0/263997959 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7fe0101f70 0x7f7fe0102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.695+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 >> 172.21.15.16:0/263997959 conn(0x7f7fe0077c60 msgr2=0x7f7fe0105520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.696+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 shutdown_connections 2024-09-21T03:07:09.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:09.697+0000 7f7fe5d2d640 1 -- 172.21.15.16:0/263997959 wait complete. 2024-09-21T03:07:10.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:07:10.674 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:07:11.164 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:10 smithi123 ceph-mon[34645]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 356 MiB/s wr, 349 op/s 2024-09-21T03:07:11.226 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:10 smithi016 ceph-mon[31253]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 356 MiB/s wr, 349 op/s 2024-09-21T03:07:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:10 smithi018 ceph-mon[34587]: pgmap v526: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 356 MiB/s wr, 349 op/s 2024-09-21T03:07:11.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.822+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3347076243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 msgr2=0x7fc3f810e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:11.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.822+0000 7fc3fedcb640 1 --2- 172.21.15.16:0/3347076243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f810e000 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8009900 tx=0x7fc3e802a420 comp rx=0 tx=0).stop 2024-09-21T03:07:11.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.824+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3347076243 shutdown_connections 2024-09-21T03:07:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.824+0000 7fc3fedcb640 1 --2- 172.21.15.16:0/3347076243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 0x7fc3f811efc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.824+0000 7fc3fedcb640 1 --2- 172.21.15.16:0/3347076243 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f810e000 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.824+0000 7fc3fedcb640 1 --2- 172.21.15.16:0/3347076243 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f810d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.824+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3347076243 >> 172.21.15.16:0/3347076243 conn(0x7fc3f806e9f0 msgr2=0x7fc3f806ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:11.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.825+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3347076243 shutdown_connections 2024-09-21T03:07:11.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.825+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3347076243 wait complete. 2024-09-21T03:07:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.826+0000 7fc3fedcb640 1 Processor -- start 2024-09-21T03:07:11.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.826+0000 7fc3fedcb640 1 -- start start 2024-09-21T03:07:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f81a9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f81a96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 0x7fc3f81ae940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:11.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f8122770 con 0x7fc3f8111cd0 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81225f0 con 0x7fc3f810dbf0 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.827+0000 7fc3fedcb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc3f81228f0 con 0x7fc3f811cba0 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.828+0000 7fc3fddc9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f81a9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.828+0000 7fc3fddc9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f81a9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:58422/0 (socket says 172.21.15.16:58422) 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.828+0000 7fc3fddc9640 1 -- 172.21.15.16:0/3681042683 learned_addr learned my addr 172.21.15.16:0/3681042683 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.828+0000 7fc3fd5c8640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f81a96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:11.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.828+0000 7fc3fe5ca640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 0x7fc3f81ae940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.829+0000 7fc3fd5c8640 1 -- 172.21.15.16:0/3681042683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 msgr2=0x7fc3f81ae940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.829+0000 7fc3fd5c8640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 0x7fc3f81ae940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.829+0000 7fc3fd5c8640 1 -- 172.21.15.16:0/3681042683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 msgr2=0x7fc3f81a9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.829+0000 7fc3fd5c8640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f81a9100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.829+0000 7fc3fd5c8640 1 -- 172.21.15.16:0/3681042683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3f81af070 con 0x7fc3f8111cd0 2024-09-21T03:07:11.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.830+0000 7fc3fd5c8640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f81a96a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8033040 tx=0x7fc3e80054f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:11.837 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.836+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e8041070 con 0x7fc3f8111cd0 2024-09-21T03:07:11.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.836+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3681042683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3f81af300 con 0x7fc3f8111cd0 2024-09-21T03:07:11.838 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.836+0000 7fc3fedcb640 1 -- 172.21.15.16:0/3681042683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3f81b5ac0 con 0x7fc3f8111cd0 2024-09-21T03:07:11.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.839+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3e80041b0 con 0x7fc3f8111cd0 2024-09-21T03:07:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.839+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3dc0052c0 con 0x7fc3f8111cd0 2024-09-21T03:07:11.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.841+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3e803dd70 con 0x7fc3f8111cd0 2024-09-21T03:07:11.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.847+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc3e8044070 con 0x7fc3f8111cd0 2024-09-21T03:07:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.849+0000 7fc3eeffd640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 0x7fc3e40b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:11.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.849+0000 7fc3fddc9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 0x7fc3e40b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:11.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.850+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7fc3e81089f0 con 0x7fc3f8111cd0 2024-09-21T03:07:11.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.851+0000 7fc3fddc9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 0x7fc3e40b8d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009a10 tx=0x7fc3f4009190 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:11.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:11.859+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fc3e80cef80 con 0x7fc3f8111cd0 2024-09-21T03:07:12.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.440+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc3dc0055b0 con 0x7fc3f8111cd0 2024-09-21T03:07:12.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.449+0000 7fc3eeffd640 1 -- 172.21.15.16:0/3681042683 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7fc3e80d3360 con 0x7fc3f8111cd0 2024-09-21T03:07:12.451 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:07:12.452 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:07:12.452 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.455+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 msgr2=0x7fc3e40b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.455+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 0x7fc3e40b8d10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fc3f4009a10 tx=0x7fc3f4009190 comp rx=0 tx=0).stop 2024-09-21T03:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.455+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 msgr2=0x7fc3f81a96a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:12.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.455+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f81a96a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7fc3e8033040 tx=0x7fc3e80054f0 comp rx=0 tx=0).stop 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 shutdown_connections 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc3e40b6800 0x7fc3e40b8d10 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc3f8111cd0 0x7fc3f81a96a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3f811cba0 0x7fc3f81ae940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 --2- 172.21.15.16:0/3681042683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc3f810dbf0 0x7fc3f81a9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.458+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 >> 172.21.15.16:0/3681042683 conn(0x7fc3f806e9f0 msgr2=0x7fc3f8110980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:12.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.459+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 shutdown_connections 2024-09-21T03:07:12.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:12.459+0000 7fc3ecff9640 1 -- 172.21.15.16:0/3681042683 wait complete. 2024-09-21T03:07:13.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:12 smithi016 ceph-mon[31253]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 972 KiB/s rd, 353 MiB/s wr, 339 op/s 2024-09-21T03:07:13.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:12 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3681042683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:13.345 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:07:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:12 smithi018 ceph-mon[34587]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 972 KiB/s rd, 353 MiB/s wr, 339 op/s 2024-09-21T03:07:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:12 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3681042683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:13.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:13 smithi123 ceph-mon[34645]: pgmap v527: 97 pgs: 97 active+clean; 60 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 972 KiB/s rd, 353 MiB/s wr, 339 op/s 2024-09-21T03:07:13.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:13 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3681042683' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:14.424 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.073+0000 7f28666f6640 1 -- 172.21.15.16:0/3609818565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2860103b80 msgr2=0x7f2860103f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.073+0000 7f28666f6640 1 --2- 172.21.15.16:0/3609818565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2860103b80 0x7f2860103f90 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f2850009900 tx=0x7f285002a410 comp rx=0 tx=0).stop 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 -- 172.21.15.16:0/3609818565 shutdown_connections 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 --2- 172.21.15.16:0/3609818565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f2860110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 --2- 172.21.15.16:0/3609818565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f28601094c0 0x7f28601098d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 --2- 172.21.15.16:0/3609818565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2860103b80 0x7f2860103f90 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 -- 172.21.15.16:0/3609818565 >> 172.21.15.16:0/3609818565 conn(0x7f28600ff610 msgr2=0x7f2860101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 -- 172.21.15.16:0/3609818565 shutdown_connections 2024-09-21T03:07:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 -- 172.21.15.16:0/3609818565 wait complete. 2024-09-21T03:07:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.074+0000 7f28666f6640 1 Processor -- start 2024-09-21T03:07:15.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 -- start start 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 0x7f286019a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f286019fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2860113ad0 con 0x7f28601094c0 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2860113950 con 0x7f2860103b80 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f28666f6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2860113c50 con 0x7f2860109e10 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44408/0 (socket says 172.21.15.16:44408) 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f2857fff640 1 -- 172.21.15.16:0/1252412981 learned_addr learned my addr 172.21.15.16:0/1252412981 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f286019fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.075+0000 7f285ffff640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 0x7f286019a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 -- 172.21.15.16:0/1252412981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 msgr2=0x7f286019a4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 0x7f286019a4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 -- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 msgr2=0x7f286019aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 -- 172.21.15.16:0/1252412981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28601a0430 con 0x7f2860109e10 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2857fff640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f285ffff640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 0x7f286019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f286019fd00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fd0 tx=0x7f285800ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f285dffb640 1 -- 172.21.15.16:0/1252412981 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f285800ce20 con 0x7f2860109e10 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28601a0720 con 0x7f2860109e10 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f285dffb640 1 -- 172.21.15.16:0/1252412981 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2858005020 con 0x7f2860109e10 2024-09-21T03:07:15.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.076+0000 7f285dffb640 1 -- 172.21.15.16:0/1252412981 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2858010690 con 0x7f2860109e10 2024-09-21T03:07:15.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.077+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28600755f0 con 0x7f2860109e10 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.079+0000 7f286cc6a740 1 Processor -- start 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.079+0000 7f285dffb640 1 -- 172.21.15.16:0/1252412981 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2858010830 con 0x7f2860109e10 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 -- start start 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac7f6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac7fbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac80120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56039abf5ea0 con 0x56039a8b7390 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56039abe52d0 con 0x56039a8af1b0 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f286cc6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56039abe90e0 con 0x56039a8d8070 2024-09-21T03:07:15.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f2857fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac7f6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f2857fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac7f6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49402/0 (socket says 172.21.15.16:49402) 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f2857fff640 1 -- 172.21.15.16:0/3212491518 learned_addr learned my addr 172.21.15.16:0/3212491518 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f2857fff640 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac80120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.080+0000 7f285ffff640 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac7fbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f285dffb640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 0x7f284c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 -- 172.21.15.16:0/3212491518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 msgr2=0x56039ac7f6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac7f6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 -- 172.21.15.16:0/3212491518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 msgr2=0x56039ac7fbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac7fbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 -- 172.21.15.16:0/3212491518 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039a943650 con 0x56039a8b7390 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2857fff640 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac80120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f28480058c0 tx=0x7f2848005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15a4842af1308d48 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f285dffb640 1 -- 172.21.15.16:0/1252412981 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f28580e7e10 con 0x7f2860109e10 2024-09-21T03:07:15.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 0x7f284c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.081+0000 7f28567fc640 1 -- 172.21.15.16:0/3212491518 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848026070 con 0x56039a8b7390 2024-09-21T03:07:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.082+0000 7f28567fc640 1 -- 172.21.15.16:0/3212491518 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848009c50 con 0x56039a8b7390 2024-09-21T03:07:15.085 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.082+0000 7f28567fc640 1 -- 172.21.15.16:0/3212491518 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848020640 con 0x56039a8b7390 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.082+0000 7f286cc6a740 1 -- 172.21.15.16:0/3212491518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 msgr2=0x56039ac80120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.082+0000 7f2864c93640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 0x7f284c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f285800c760 tx=0x7f2858014000 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.082+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac80120 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f28480058c0 tx=0x7f2848005f90 comp rx=0 tx=0).stop 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 -- 172.21.15.16:0/3212491518 shutdown_connections 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac80120 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac7fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3212491518 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac7f6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 -- 172.21.15.16:0/3212491518 >> 172.21.15.16:0/3212491518 conn(0x56039a8cc460 msgr2=0x56039ac7d590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 -- 172.21.15.16:0/3212491518 shutdown_connections 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 -- 172.21.15.16:0/3212491518 wait complete. 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.083+0000 7f286cc6a740 1 -- start start 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 Processor -- start 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac97170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56039aa46700 con 0x56039a8b7390 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56039ab80160 con 0x56039a8af1b0 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56039ac8de80 con 0x56039a8d8070 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f286cc6a740 10 client.0 _ll_register_callbacks cb 0x56039ab999e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:15.089 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f285ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f285ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44436/0 (socket says 172.21.15.16:44436) 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f285ffff640 1 -- 172.21.15.16:0/4141909424 learned_addr learned my addr 172.21.15.16:0/4141909424 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f2857fff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.084+0000 7f2857fff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac97170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 -- 172.21.15.16:0/4141909424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 msgr2=0x56039ac97170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac97170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 -- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 msgr2=0x56039ac8ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 -- 172.21.15.16:0/4141909424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039abadc90 con 0x56039a8b7390 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f2857fff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac97170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f285ffff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2850009900 tx=0x7f28500316b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef24fed90f35d33 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f283f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850035040 con 0x56039a8b7390 2024-09-21T03:07:15.090 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2848009c50 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.085+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f2850004240 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f28500318e0 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 10 client.16158 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56039ac99c60 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2848009c50 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2850004aa0 con 0x56039a8b7390 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 10 client.16158 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 10 client.16158 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 20 client.16158 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a8969f0 0x56039ac9d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 4 client.16158 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.086+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56039abf7df0 con 0x56039a8969f0 2024-09-21T03:07:15.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.087+0000 7f2857fff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a8969f0 0x56039ac9d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.087+0000 7f283f7fe640 10 client.16158 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.087+0000 7f2857fff640 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a8969f0 0x56039ac9d0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fb21a7c8d0b6a5d server_cookie=497cb3580dd6454 in_seq=0 out_seq=0 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f283f7fe640 1 -- 172.21.15.16:0/4141909424 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56039abf7df0 con 0x56039a8969f0 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f283f7fe640 10 client.16158 handle_command_reply: tid=0 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 client.16158 shutdown 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 msgr2=0x56039ac8b320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f2850009900 tx=0x7f28500316b0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 shutdown_connections 2024-09-21T03:07:15.093 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a8969f0 0x56039ac9d0a0 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8b7390 0x56039ac8b320 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac97170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 --2- 172.21.15.16:0/4141909424 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 >> 172.21.15.16:0/4141909424 conn(0x56039a8cc460 msgr2=0x56039ac82880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f283f7fe640 0 client.16158 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.088+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 shutdown_connections 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.089+0000 7f286cc6a740 1 -- 172.21.15.16:0/4141909424 wait complete. 2024-09-21T03:07:15.094 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.089+0000 7f286cc6a740 20 client.16158 trim_cache size 0 max 16384 2024-09-21T03:07:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.102+0000 7f286cc6a740 1 Processor -- start 2024-09-21T03:07:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 -- start start 2024-09-21T03:07:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 0x56039ac8ab90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 0x56039ac8b0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.105 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac827a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ab5f0 con 0x56039a8af1b0 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ade50 con 0x56039a8b7390 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f286cc6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28601abc40 con 0x56039a8d8070 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 0x56039ac8ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 0x56039ac8b0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 0x56039ac8ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44450/0 (socket says 172.21.15.16:44450) 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f2857fff640 1 -- 172.21.15.16:0/3869214075 learned_addr learned my addr 172.21.15.16:0/3869214075 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f2857fff640 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac827a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 -- 172.21.15.16:0/3869214075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 msgr2=0x56039ac827a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac827a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.106 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 -- 172.21.15.16:0/3869214075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 msgr2=0x56039ac8ab90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 0x56039ac8ab90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.103+0000 7f285ffff640 1 -- 172.21.15.16:0/3869214075 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56039ac81990 con 0x56039a8b7390 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.104+0000 7f285ffff640 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 0x56039ac8b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2850006010 tx=0x7f28500046e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55953bb01aafaf42 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.104+0000 7f2855ffb640 1 -- 172.21.15.16:0/3869214075 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850035070 con 0x56039a8b7390 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.104+0000 7f2855ffb640 1 -- 172.21.15.16:0/3869214075 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2850004fc0 con 0x56039a8b7390 2024-09-21T03:07:15.107 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.104+0000 7f2855ffb640 1 -- 172.21.15.16:0/3869214075 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2850000e60 con 0x56039a8b7390 2024-09-21T03:07:15.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- 172.21.15.16:0/3869214075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 msgr2=0x56039ac8b0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 0x56039ac8b0d0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f2850006010 tx=0x7f28500046e0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- 172.21.15.16:0/3869214075 shutdown_connections 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8d8070 0x56039ac827a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8b7390 0x56039ac8b0d0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 --2- 172.21.15.16:0/3869214075 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8af1b0 0x56039ac8ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- 172.21.15.16:0/3869214075 >> 172.21.15.16:0/3869214075 conn(0x56039a8cc460 msgr2=0x56039ac89660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- 172.21.15.16:0/3869214075 shutdown_connections 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- 172.21.15.16:0/3869214075 wait complete. 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.105+0000 7f286cc6a740 1 -- start start 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 Processor -- start 2024-09-21T03:07:15.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8b7390 0x56039ac8ef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ac310 con 0x56039a8d8070 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ad780 con 0x56039a8af1b0 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f28601ac9e0 con 0x56039a8b7390 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f286cc6a740 10 client.0 _ll_register_callbacks cb 0x7f2850009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f2857fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44452/0 (socket says 172.21.15.16:44452) 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f2857fff640 1 -- 172.21.15.16:0/1508657019 learned_addr learned my addr 172.21.15.16:0/1508657019 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.106+0000 7f285ffff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8b7390 0x56039ac8ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 -- 172.21.15.16:0/1508657019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8b7390 msgr2=0x56039ac8ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8b7390 0x56039ac8ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 -- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 msgr2=0x56039ac8e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 -- 172.21.15.16:0/1508657019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56039ad06150 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f2857fff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2848012fd0 tx=0x7f2848021d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6247bddf1f4547 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f283f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2848026040 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(80..80 src has 1..80) ==== 8580+0+0 (secure 0 0 0) 0x7f2848005050 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2850004fc0 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2848005250 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f286cc6a740 10 client.16164 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.107+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56039ad073d0 con 0x56039a8d8070 2024-09-21T03:07:15.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2850004fc0 con 0x56039a8d8070 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2848043030 con 0x56039a8d8070 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 10 client.16164 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 10 client.16164 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 20 client.16164 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a86bda0 0x56039ad099a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 4 client.16164 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56039abf38a0 con 0x56039a86bda0 2024-09-21T03:07:15.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.108+0000 7f285ffff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a86bda0 0x56039ad099a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f283f7fe640 10 client.16164 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f285ffff640 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a86bda0 0x56039ad099a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22557164a5b9db11 server_cookie=172138112f4b82f0 in_seq=0 out_seq=0 2024-09-21T03:07:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f283f7fe640 1 -- 172.21.15.16:0/1508657019 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56039abf38a0 con 0x56039a86bda0 2024-09-21T03:07:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f283f7fe640 10 client.16164 handle_command_reply: tid=0 2024-09-21T03:07:15.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 client.16164 shutdown 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 msgr2=0x56039ad05ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f2848012fd0 tx=0x7f2848021d70 comp rx=0 tx=0).stop 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 shutdown_connections 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56039a86bda0 0x56039ad099a0 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56039a8d8070 0x56039ad05ae0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56039a8b7390 0x56039ac8ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f283f7fe640 0 client.16164 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 --2- 172.21.15.16:0/1508657019 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56039a8af1b0 0x56039ac8e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.109+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 >> 172.21.15.16:0/1508657019 conn(0x56039a8cc460 msgr2=0x56039ac827a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 shutdown_connections 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f286cc6a740 1 -- 172.21.15.16:0/1508657019 wait complete. 2024-09-21T03:07:15.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f286cc6a740 20 client.16164 trim_cache size 0 max 16384 2024-09-21T03:07:15.116 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:07:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 msgr2=0x7f284c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 0x7f284c0b8b90 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f285800c760 tx=0x7f2858014000 comp rx=0 tx=0).stop 2024-09-21T03:07:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 msgr2=0x7f286019fd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:15.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f286019fd00 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f2858009fd0 tx=0x7f285800ef10 comp rx=0 tx=0).stop 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 shutdown_connections 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f284c0b6680 0x7f284c0b8b90 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f28601094c0 0x7f286019aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2860109e10 0x7f286019fd00 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 --2- 172.21.15.16:0/1252412981 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2860103b80 0x7f286019a4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 >> 172.21.15.16:0/1252412981 conn(0x7f28600ff610 msgr2=0x7f2860101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.110+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 shutdown_connections 2024-09-21T03:07:15.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:15.111+0000 7f28666f6640 1 -- 172.21.15.16:0/1252412981 wait complete. 2024-09-21T03:07:15.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:15 smithi018 ceph-mon[34587]: pgmap v528: 97 pgs: 97 active+clean; 61 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1016 KiB/s rd, 362 MiB/s wr, 459 op/s 2024-09-21T03:07:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:15 smithi123 ceph-mon[34645]: pgmap v528: 97 pgs: 97 active+clean; 61 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1016 KiB/s rd, 362 MiB/s wr, 459 op/s 2024-09-21T03:07:15.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:15 smithi016 ceph-mon[31253]: pgmap v528: 97 pgs: 97 active+clean; 61 GiB data, 184 GiB used, 889 GiB / 1.0 TiB avail; 1016 KiB/s rd, 362 MiB/s wr, 459 op/s 2024-09-21T03:07:16.883 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:07:16.883 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:07:17.172 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:17 smithi016 ceph-mon[31253]: pgmap v529: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 885 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 442 MiB/s wr, 1.42k op/s 2024-09-21T03:07:17.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:17 smithi016 ceph-mon[31253]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:07:17.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:17 smithi018 ceph-mon[34587]: pgmap v529: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 885 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 442 MiB/s wr, 1.42k op/s 2024-09-21T03:07:17.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:17 smithi018 ceph-mon[34587]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:07:17.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:17 smithi123 ceph-mon[34645]: pgmap v529: 97 pgs: 97 active+clean; 61 GiB data, 187 GiB used, 885 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 442 MiB/s wr, 1.42k op/s 2024-09-21T03:07:17.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:17 smithi123 ceph-mon[34645]: osdmap e81: 12 total, 12 up, 12 in 2024-09-21T03:07:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.767+0000 7f264f8aa640 1 -- 172.21.15.16:0/1016659018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648109570 msgr2=0x7f2648109980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.807 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 --2- 172.21.15.16:0/1016659018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648109570 0x7f2648109980 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f2638009900 tx=0x7f263802a410 comp rx=0 tx=0).stop 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 -- 172.21.15.16:0/1016659018 shutdown_connections 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 --2- 172.21.15.16:0/1016659018 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648109570 0x7f2648109980 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 --2- 172.21.15.16:0/1016659018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f26481106f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 --2- 172.21.15.16:0/1016659018 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648103c30 0x7f2648104040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 -- 172.21.15.16:0/1016659018 >> 172.21.15.16:0/1016659018 conn(0x7f264807b8d0 msgr2=0x7f264807bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 -- 172.21.15.16:0/1016659018 shutdown_connections 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.768+0000 7f264f8aa640 1 -- 172.21.15.16:0/1016659018 wait complete. 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 Processor -- start 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 -- start start 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 0x7f264819aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f264819fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.808 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2648113ae0 con 0x7f2648103c30 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2648113960 con 0x7f2648109570 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.769+0000 7f264f8aa640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2648113c60 con 0x7f2648109ec0 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264d646640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264d646640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44482/0 (socket says 172.21.15.16:44482) 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264d646640 1 -- 172.21.15.16:0/124399833 learned_addr learned my addr 172.21.15.16:0/124399833 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f264819fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264ce45640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 0x7f264819aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 -- 172.21.15.16:0/124399833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 msgr2=0x7f264819aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 0x7f264819aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 -- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 msgr2=0x7f264819a490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 -- 172.21.15.16:0/124399833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26481a0400 con 0x7f2648109ec0 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264d646640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264ce45640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 0x7f264819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:17.809 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.770+0000 7f264de47640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f264819fcd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2640009fd0 tx=0x7f264000ef10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.771+0000 7f263e7fc640 1 -- 172.21.15.16:0/124399833 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264000ce20 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.771+0000 7f263e7fc640 1 -- 172.21.15.16:0/124399833 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2640005020 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.771+0000 7f263e7fc640 1 -- 172.21.15.16:0/124399833 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2640010690 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.771+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26481a06f0 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.771+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26480755f0 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.774+0000 7f263e7fc640 1 -- 172.21.15.16:0/124399833 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f26400054d0 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.775+0000 7f263e7fc640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 0x7f26340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.776+0000 7f264d646640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 0x7f26340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.776+0000 7f263e7fc640 1 -- 172.21.15.16:0/124399833 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f26400e7d80 con 0x7f2648109ec0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.776+0000 7f264d646640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 0x7f26340b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2644005fd0 tx=0x7f264400b4a0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.777+0000 7f2655e1e740 1 Processor -- start 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.777+0000 7f2655e1e740 1 -- start start 2024-09-21T03:07:17.810 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.777+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4306970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4306eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f2655e1e740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff427ce60 con 0x55dff3f5f350 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f2655e1e740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dff426c290 con 0x55dff3f36490 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f264d646640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f2655e1e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dff42700a0 con 0x55dff3f3e670 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f264d646640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44490/0 (socket says 172.21.15.16:44490) 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.778+0000 7f264d646640 1 -- 172.21.15.16:0/2688621659 learned_addr learned my addr 172.21.15.16:0/2688621659 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264ce45640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4306970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264de47640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4306eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264d646640 1 -- 172.21.15.16:0/2688621659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 msgr2=0x55dff4306eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264d646640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4306eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264d646640 1 -- 172.21.15.16:0/2688621659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 msgr2=0x55dff4306970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.811 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264d646640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4306970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264d646640 1 -- 172.21.15.16:0/2688621659 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff3fca930 con 0x55dff3f5f350 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.779+0000 7f264de47640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4306eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.780+0000 7f264d646640 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2644011920 tx=0x7f2644013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78dd88544d002a0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.780+0000 7f26337fe640 1 -- 172.21.15.16:0/2688621659 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264400eb50 con 0x55dff3f5f350 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.780+0000 7f26337fe640 1 -- 172.21.15.16:0/2688621659 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f264400ecf0 con 0x55dff3f5f350 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.780+0000 7f26337fe640 1 -- 172.21.15.16:0/2688621659 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f264401c610 con 0x55dff3f5f350 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.782+0000 7f2655e1e740 1 -- 172.21.15.16:0/2688621659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 msgr2=0x55dff4306430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.782+0000 7f2655e1e740 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f2644011920 tx=0x7f2644013000 comp rx=0 tx=0).stop 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 -- 172.21.15.16:0/2688621659 shutdown_connections 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4306eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4306970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 --2- 172.21.15.16:0/2688621659 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4306430 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 -- 172.21.15.16:0/2688621659 >> 172.21.15.16:0/2688621659 conn(0x55dff3f53740 msgr2=0x55dff4304320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.812 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 -- 172.21.15.16:0/2688621659 shutdown_connections 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.783+0000 7f2655e1e740 1 -- 172.21.15.16:0/2688621659 wait complete. 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.784+0000 7f2655e1e740 1 -- start start 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.784+0000 7f2655e1e740 1 Processor -- start 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.785+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.786+0000 7f264d646640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.786+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43125f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f264d646640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49482/0 (socket says 172.21.15.16:49482) 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f264d646640 1 -- 172.21.15.16:0/1379594590 learned_addr learned my addr 172.21.15.16:0/1379594590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431df00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55dff40cd890 con 0x55dff3f3e670 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55dff4206d40 con 0x55dff3f5f350 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55dff4314c10 con 0x55dff3f36490 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.787+0000 7f264ce45640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43125f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264d646640 1 -- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 msgr2=0x55dff431df00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264d646640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431df00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264d646640 1 -- 172.21.15.16:0/1379594590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 msgr2=0x55dff43125f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f2655e1e740 10 client.0 _ll_register_callbacks cb 0x55dff426ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264d646640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43125f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264d646640 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff431e540 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f264ce45640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.788+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f264400ecf0 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.789+0000 7f264d646640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f264401e450 tx=0x7f264401e480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=524413c09dca9220 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.789+0000 7f2630ff9640 10 client.25819 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.790+0000 7f2630ff9640 1 -- 172.21.15.16:0/1379594590 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2644034060 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.790+0000 7f2630ff9640 1 -- 172.21.15.16:0/1379594590 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f2644005030 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.794+0000 7f2630ff9640 1 -- 172.21.15.16:0/1379594590 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2644005230 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.794+0000 7f2655e1e740 10 client.25819 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.795+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55dff4320cc0 con 0x55dff3f36490 2024-09-21T03:07:17.814 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.795+0000 7f2630ff9640 1 -- 172.21.15.16:0/1379594590 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f26440053d0 con 0x55dff3f36490 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.795+0000 7f2655e1e740 10 client.25819 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.795+0000 7f2655e1e740 10 client.25819 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.795+0000 7f2655e1e740 20 client.25819 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.796+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3f1dcd0 0x55dff4324020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.796+0000 7f2655e1e740 4 client.25819 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.796+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55dff427edb0 con 0x55dff3f1dcd0 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.796+0000 7f264ce45640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3f1dcd0 0x55dff4324020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.798+0000 7f264ce45640 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3f1dcd0 0x55dff4324020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b00d9409d6b0aaab server_cookie=532cba06ee20b6f0 in_seq=0 out_seq=0 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.798+0000 7f2630ff9640 10 client.25819 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.800+0000 7f2630ff9640 1 -- 172.21.15.16:0/1379594590 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55dff427edb0 con 0x55dff3f1dcd0 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.800+0000 7f2630ff9640 10 client.25819 handle_command_reply: tid=0 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.800+0000 7f2655e1e740 1 client.25819 shutdown 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 msgr2=0x55dff4312050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.815 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f264401e450 tx=0x7f264401e480 comp rx=0 tx=0).stop 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 shutdown_connections 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3f1dcd0 0x55dff4324020 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2630ff9640 0 client.25819 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43125f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431df00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 --2- 172.21.15.16:0/1379594590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff4312050 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.801+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 >> 172.21.15.16:0/1379594590 conn(0x55dff3f53740 msgr2=0x55dff4309610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.802+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 shutdown_connections 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.802+0000 7f2655e1e740 1 -- 172.21.15.16:0/1379594590 wait complete. 2024-09-21T03:07:17.816 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.803+0000 7f2655e1e740 20 client.25819 trim_cache size 0 max 16384 2024-09-21T03:07:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.843+0000 7f2655e1e740 1 Processor -- start 2024-09-21T03:07:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.843+0000 7f2655e1e740 1 -- start start 2024-09-21T03:07:17.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.844+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff430bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.844+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43116e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.845+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.845+0000 7f264d646640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff430bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.845+0000 7f264d646640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff430bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49496/0 (socket says 172.21.15.16:49496) 2024-09-21T03:07:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.845+0000 7f264d646640 1 -- 172.21.15.16:0/572241768 learned_addr learned my addr 172.21.15.16:0/572241768 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:17.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.846+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26481af990 con 0x55dff3f3e670 2024-09-21T03:07:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.846+0000 7f264de47640 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.846+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b21f0 con 0x55dff3f5f350 2024-09-21T03:07:17.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.846+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26481affe0 con 0x55dff3f36490 2024-09-21T03:07:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.847+0000 7f264de47640 1 -- 172.21.15.16:0/572241768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 msgr2=0x55dff430bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.847+0000 7f264de47640 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff430bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.847+0000 7f264de47640 1 -- 172.21.15.16:0/572241768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 msgr2=0x55dff43116e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.847+0000 7f264de47640 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43116e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.847+0000 7f264de47640 1 -- 172.21.15.16:0/572241768 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55dff4308720 con 0x55dff3f5f350 2024-09-21T03:07:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.848+0000 7f264de47640 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431de70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26400e54c0 tx=0x7f26400ad5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7a3bf15c01d79e38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.849+0000 7f2632ffd640 1 -- 172.21.15.16:0/572241768 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400d5170 con 0x55dff3f5f350 2024-09-21T03:07:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.849+0000 7f2632ffd640 1 -- 172.21.15.16:0/572241768 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26400ae2c0 con 0x55dff3f5f350 2024-09-21T03:07:17.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.849+0000 7f2632ffd640 1 -- 172.21.15.16:0/572241768 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26400a9e30 con 0x55dff3f5f350 2024-09-21T03:07:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.852+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 msgr2=0x55dff431de70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.852+0000 7f2655e1e740 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431de70 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f26400e54c0 tx=0x7f26400ad5d0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.853+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 shutdown_connections 2024-09-21T03:07:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.853+0000 7f2655e1e740 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f3e670 0x55dff43116e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.853+0000 7f2655e1e740 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f5f350 0x55dff431de70 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.854+0000 7f2655e1e740 1 --2- 172.21.15.16:0/572241768 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f36490 0x55dff430bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.854+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 >> 172.21.15.16:0/572241768 conn(0x55dff3f53740 msgr2=0x55dff430b540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.854+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 shutdown_connections 2024-09-21T03:07:17.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.855+0000 7f2655e1e740 1 -- 172.21.15.16:0/572241768 wait complete. 2024-09-21T03:07:17.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.856+0000 7f2655e1e740 1 -- start start 2024-09-21T03:07:17.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.856+0000 7f2655e1e740 1 Processor -- start 2024-09-21T03:07:17.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.858+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4325240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.858+0000 7f2655e1e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4325780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.859+0000 7f264d646640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4325240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.859+0000 7f264d646640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4325240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:54136/0 (socket says 172.21.15.16:54136) 2024-09-21T03:07:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.860+0000 7f264d646640 1 -- 172.21.15.16:0/3565308772 learned_addr learned my addr 172.21.15.16:0/3565308772 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:17.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.861+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4325cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.861+0000 7f264ce45640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4325780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.862+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b06b0 con 0x55dff3f5f350 2024-09-21T03:07:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.862+0000 7f264de47640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4325cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.863+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b1b20 con 0x55dff3f36490 2024-09-21T03:07:17.865 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.863+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f26481b0d80 con 0x55dff3f3e670 2024-09-21T03:07:17.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.864+0000 7f2655e1e740 10 client.0 _ll_register_callbacks cb 0x7f2644011820 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:17.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.865+0000 7f264ce45640 1 -- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 msgr2=0x55dff4325240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.865+0000 7f264ce45640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4325240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.865+0000 7f264ce45640 1 -- 172.21.15.16:0/3565308772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 msgr2=0x55dff4325cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.865+0000 7f264ce45640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4325cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.865+0000 7f264ce45640 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55dff438d9a0 con 0x55dff3f3e670 2024-09-21T03:07:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.867+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f26400ae2c0 con 0x55dff3f3e670 2024-09-21T03:07:17.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.867+0000 7f264ce45640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4325780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f263802a930 tx=0x7f26380041a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18ffce02205fc01 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:17.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.871+0000 7f2630ff9640 10 client.25825 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:07:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.871+0000 7f2630ff9640 1 -- 172.21.15.16:0/3565308772 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263804a020 con 0x55dff3f3e670 2024-09-21T03:07:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.871+0000 7f2630ff9640 1 -- 172.21.15.16:0/3565308772 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f2638005240 con 0x55dff3f3e670 2024-09-21T03:07:17.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.871+0000 7f2630ff9640 1 -- 172.21.15.16:0/3565308772 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2638035040 con 0x55dff3f3e670 2024-09-21T03:07:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.872+0000 7f2655e1e740 10 client.25825 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:17.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.873+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55dff438eb80 con 0x55dff3f3e670 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2630ff9640 1 -- 172.21.15.16:0/3565308772 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f26380054a0 con 0x55dff3f3e670 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 10 client.25825 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 10 client.25825 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 20 client.25825 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3ef3080 0x55dff4391150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 4 client.25825 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:07:17.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.877+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x7f26400ae2c0 con 0x55dff3ef3080 2024-09-21T03:07:17.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.881+0000 7f264d646640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3ef3080 0x55dff4391150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.884+0000 7f2630ff9640 10 client.25825 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:17.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.884+0000 7f264d646640 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3ef3080 0x55dff4391150 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c955d1191ce30b6 server_cookie=692fa36848652833 in_seq=0 out_seq=0 2024-09-21T03:07:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.886+0000 7f2630ff9640 1 -- 172.21.15.16:0/3565308772 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x7f26400ae2c0 con 0x55dff3ef3080 2024-09-21T03:07:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.886+0000 7f2630ff9640 10 client.25825 handle_command_reply: tid=0 2024-09-21T03:07:17.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.886+0000 7f2655e1e740 1 client.25825 shutdown 2024-09-21T03:07:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 msgr2=0x55dff4325780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4325780 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f263802a930 tx=0x7f26380041a0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 shutdown_connections 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55dff3ef3080 0x55dff4391150 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55dff3f5f350 0x55dff4325cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55dff3f3e670 0x55dff4325780 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 --2- 172.21.15.16:0/3565308772 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55dff3f36490 0x55dff4325240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.893+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 >> 172.21.15.16:0/3565308772 conn(0x55dff3f53740 msgr2=0x55dff4307460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.894+0000 7f2630ff9640 0 client.25825 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.894+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 shutdown_connections 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.894+0000 7f2655e1e740 1 -- 172.21.15.16:0/3565308772 wait complete. 2024-09-21T03:07:17.896 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.894+0000 7f2655e1e740 20 client.25825 trim_cache size 0 max 16384 2024-09-21T03:07:17.897 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:07:17.897 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:07:17.897 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "66226652-4934-4aed-bd6e-225c12fed8ba", 2024-09-21T03:07:17.897 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:07:17.897 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 msgr2=0x7f26340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 0x7f26340b8b90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f2644005fd0 tx=0x7f264400b4a0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 msgr2=0x7f264819fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f264819fcd0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2640009fd0 tx=0x7f264000ef10 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 shutdown_connections 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26340b6680 0x7f26340b8b90 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2648109ec0 0x7f264819fcd0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2648109570 0x7f264819aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 --2- 172.21.15.16:0/124399833 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2648103c30 0x7f264819a490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:17.899 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.897+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 >> 172.21.15.16:0/124399833 conn(0x7f264807b8d0 msgr2=0x7f2648107600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:17.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.898+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 shutdown_connections 2024-09-21T03:07:17.900 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:17.898+0000 7f264f8aa640 1 -- 172.21.15.16:0/124399833 wait complete. 2024-09-21T03:07:18.950 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:07:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:19 smithi018 ceph-mon[34587]: pgmap v531: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 487 MiB/s wr, 2.15k op/s 2024-09-21T03:07:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:19 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:07:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:19 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:19 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:07:19.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:19 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:19 smithi123 ceph-mon[34645]: pgmap v531: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 487 MiB/s wr, 2.15k op/s 2024-09-21T03:07:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:19 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:07:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:19 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:19 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:07:19.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:19 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:19 smithi016 ceph-mon[31253]: pgmap v531: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 487 MiB/s wr, 2.15k op/s 2024-09-21T03:07:19.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:19 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:07:19.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:19 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:19 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:07:19.517 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:19 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:07:19.773 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:20.464 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:20 smithi016 ceph-mon[31253]: pgmap v532: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 481 MiB/s wr, 2.38k op/s 2024-09-21T03:07:20.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.626+0000 7fb20d042640 1 -- 172.21.15.16:0/3188055351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 msgr2=0x7fb208110660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.627+0000 7fb20d042640 1 --2- 172.21.15.16:0/3188055351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208110660 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fb200009ca0 tx=0x7fb20002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.628+0000 7fb20d042640 1 -- 172.21.15.16:0/3188055351 shutdown_connections 2024-09-21T03:07:20.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.628+0000 7fb20d042640 1 --2- 172.21.15.16:0/3188055351 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208110660 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.628+0000 7fb20d042640 1 --2- 172.21.15.16:0/3188055351 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb2081094e0 0x7fb2081098f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.628+0000 7fb20d042640 1 --2- 172.21.15.16:0/3188055351 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb208103ba0 0x7fb208103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.635 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.628+0000 7fb20d042640 1 -- 172.21.15.16:0/3188055351 >> 172.21.15.16:0/3188055351 conn(0x7fb2080ff630 msgr2=0x7fb208101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.635+0000 7fb20d042640 1 -- 172.21.15.16:0/3188055351 shutdown_connections 2024-09-21T03:07:20.640 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.635+0000 7fb20d042640 1 -- 172.21.15.16:0/3188055351 wait complete. 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.637+0000 7fb20d042640 1 Processor -- start 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.637+0000 7fb20d042640 1 -- start start 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.637+0000 7fb20d042640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 0x7fb208199010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb20d042640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081094e0 0x7fb208199550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb20d042640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208199a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb20d042640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb208113b40 con 0x7fb208109e30 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb20d042640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb2081139c0 con 0x7fb208103ba0 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb20d042640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb208113cc0 con 0x7fb2081094e0 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 0x7fb208199010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 0x7fb208199010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:54160/0 (socket says 172.21.15.16:54160) 2024-09-21T03:07:20.641 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb206d9d640 1 -- 172.21.15.16:0/716990929 learned_addr learned my addr 172.21.15.16:0/716990929 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.638+0000 7fb1fffff640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081094e0 0x7fb208199550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208199a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 -- 172.21.15.16:0/716990929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081094e0 msgr2=0x7fb208199550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081094e0 0x7fb208199550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 -- 172.21.15.16:0/716990929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 msgr2=0x7fb208199010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 0x7fb208199010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.639+0000 7fb20759e640 1 -- 172.21.15.16:0/716990929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb2081a02b0 con 0x7fb208109e30 2024-09-21T03:07:20.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.640+0000 7fb20759e640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208199a90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb200005e30 tx=0x7fb2000319f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.641+0000 7fb204d99640 1 -- 172.21.15.16:0/716990929 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb200003bf0 con 0x7fb208109e30 2024-09-21T03:07:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.641+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb2081a0540 con 0x7fb208109e30 2024-09-21T03:07:20.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.641+0000 7fb204d99640 1 -- 172.21.15.16:0/716990929 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb200003d90 con 0x7fb208109e30 2024-09-21T03:07:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.641+0000 7fb204d99640 1 -- 172.21.15.16:0/716990929 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb20002fba0 con 0x7fb208109e30 2024-09-21T03:07:20.644 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.641+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb2081a0a80 con 0x7fb208109e30 2024-09-21T03:07:20.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.647+0000 7fb2135b6740 1 Processor -- start 2024-09-21T03:07:20.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.650+0000 7fb204d99640 1 -- 172.21.15.16:0/716990929 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb20002fd40 con 0x7fb208109e30 2024-09-21T03:07:20.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.652+0000 7fb2135b6740 1 -- start start 2024-09-21T03:07:20.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.653+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6d94650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb204d99640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 0x7fb1f40b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb204d99640 1 -- 172.21.15.16:0/716990929 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7fb200108830 con 0x7fb208109e30 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb206d9d640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 0x7fb1f40b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 0x5558c6d94b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6d94650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d950d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb2135b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558c6d0ae50 con 0x5558c69c41b0 2024-09-21T03:07:20.662 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb2135b6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5558c6cfa280 con 0x5558c69ed070 2024-09-21T03:07:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.655+0000 7fb2135b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558c6cfe090 con 0x5558c69cc390 2024-09-21T03:07:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.660+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 0x5558c6d94b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.660+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6d94650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:54170/0 (socket says 172.21.15.16:54170) 2024-09-21T03:07:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.660+0000 7fb1fffff640 1 -- 172.21.15.16:0/3711664530 learned_addr learned my addr 172.21.15.16:0/3711664530 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:20.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.661+0000 7fb1fffff640 1 -- 172.21.15.16:0/3711664530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 msgr2=0x5558c6d950d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.661+0000 7fb1fffff640 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d950d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.661+0000 7fb1fffff640 1 -- 172.21.15.16:0/3711664530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 msgr2=0x5558c6d94650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.661+0000 7fb1fffff640 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6d94650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.661+0000 7fb1fffff640 1 -- 172.21.15.16:0/3711664530 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c6a58650 con 0x5558c69c41b0 2024-09-21T03:07:20.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.662+0000 7fb206d9d640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 0x7fb1f40b8d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb208110620 tx=0x7fb1f800b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.664+0000 7fb1fffff640 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 0x5558c6d94b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0012730 tx=0x7fb1f0004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b4c610239b91259 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.665+0000 7fb1fd7fa640 1 -- 172.21.15.16:0/3711664530 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0012cb0 con 0x5558c69c41b0 2024-09-21T03:07:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.666+0000 7fb1fd7fa640 1 -- 172.21.15.16:0/3711664530 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f0012e50 con 0x5558c69c41b0 2024-09-21T03:07:20.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.666+0000 7fb1fd7fa640 1 -- 172.21.15.16:0/3711664530 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00035f0 con 0x5558c69c41b0 2024-09-21T03:07:20.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.667+0000 7fb2135b6740 1 -- 172.21.15.16:0/3711664530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 msgr2=0x5558c6d94b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.667+0000 7fb2135b6740 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 0x5558c6d94b90 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0012730 tx=0x7fb1f0004730 comp rx=0 tx=0).stop 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 -- 172.21.15.16:0/3711664530 shutdown_connections 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69c41b0 0x5558c6d94b90 secure :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0012730 tx=0x7fb1f0004730 comp rx=0 tx=0).stop 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d950d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 --2- 172.21.15.16:0/3711664530 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6d94650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 -- 172.21.15.16:0/3711664530 >> 172.21.15.16:0/3711664530 conn(0x5558c69e1460 msgr2=0x5558c6d92540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 -- 172.21.15.16:0/3711664530 shutdown_connections 2024-09-21T03:07:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.668+0000 7fb2135b6740 1 -- 172.21.15.16:0/3711664530 wait complete. 2024-09-21T03:07:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 -- start start 2024-09-21T03:07:20.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 Processor -- start 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69cc390 0x5558c6d9ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ab9f0 0x5558c6dac120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5558c6b5b700 con 0x5558c69cc390 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5558c6c95110 con 0x5558c69ab9f0 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5558c6da2e30 con 0x5558c69ed070 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.669+0000 7fb2135b6740 10 client.0 _ll_register_callbacks cb 0x5558c6cae990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.670+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.670+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49538/0 (socket says 172.21.15.16:49538) 2024-09-21T03:07:20.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.670+0000 7fb1fffff640 1 -- 172.21.15.16:0/103305314 learned_addr learned my addr 172.21.15.16:0/103305314 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.671+0000 7fb1fffff640 1 -- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ab9f0 msgr2=0x5558c6dac120 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.671+0000 7fb1fffff640 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ab9f0 0x5558c6dac120 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.671+0000 7fb1fffff640 1 -- 172.21.15.16:0/103305314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69cc390 msgr2=0x5558c6d9ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.671+0000 7fb1fffff640 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69cc390 0x5558c6d9ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.671+0000 7fb1fffff640 1 -- 172.21.15.16:0/103305314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c6cc2c40 con 0x5558c69ed070 2024-09-21T03:07:20.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.672+0000 7fb1fffff640 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005d00 tx=0x7fb1f0005fd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a2944d69627dcf4e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.674+0000 7fb1ee7fc640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:07:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.674+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0012cb0 con 0x5558c69ed070 2024-09-21T03:07:20.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.674+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7fb1f0004220 con 0x5558c69ed070 2024-09-21T03:07:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.674+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f0012e50 con 0x5558c69ed070 2024-09-21T03:07:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.675+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1f000fe60 con 0x5558c69ed070 2024-09-21T03:07:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.675+0000 7fb2135b6740 10 client.25834 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:20.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.675+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5558c6daecf0 con 0x5558c69ed070 2024-09-21T03:07:20.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.675+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f0012e50 con 0x5558c69ed070 2024-09-21T03:07:20.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb1f0031480 con 0x5558c69ed070 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 10 client.25834 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 10 client.25834 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 20 client.25834 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6980da0 0x5558c6db2480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 4 client.25834 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:20.678 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.676+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5558c6d0cda0 con 0x5558c6980da0 2024-09-21T03:07:20.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:20 smithi018 ceph-mon[34587]: pgmap v532: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 481 MiB/s wr, 2.38k op/s 2024-09-21T03:07:20.683 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.681+0000 7fb206d9d640 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6980da0 0x5558c6db2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.693+0000 7fb206d9d640 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6980da0 0x5558c6db2480 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8de6f8f5362bb2eb server_cookie=bbc93df742e45ddb in_seq=0 out_seq=0 2024-09-21T03:07:20.694 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.693+0000 7fb1ee7fc640 10 client.25834 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.695+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/103305314 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5558c6d0cda0 con 0x5558c6980da0 2024-09-21T03:07:20.696 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.695+0000 7fb1ee7fc640 10 client.25834 handle_command_reply: tid=0 2024-09-21T03:07:20.697 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.695+0000 7fb2135b6740 1 client.25834 shutdown 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 msgr2=0x5558c6da04f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb1f0005d00 tx=0x7fb1f0005fd0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 shutdown_connections 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6980da0 0x5558c6db2480 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ab9f0 0x5558c6dac120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ed070 0x5558c6da04f0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 --2- 172.21.15.16:0/103305314 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69cc390 0x5558c6d9ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.699+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 >> 172.21.15.16:0/103305314 conn(0x5558c69e1460 msgr2=0x5558c6d97830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.700+0000 7fb1ee7fc640 0 client.25834 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.701+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 shutdown_connections 2024-09-21T03:07:20.702 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.701+0000 7fb2135b6740 1 -- 172.21.15.16:0/103305314 wait complete. 2024-09-21T03:07:20.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.701+0000 7fb2135b6740 20 client.25834 trim_cache size 0 max 16384 2024-09-21T03:07:20.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:20 smithi123 ceph-mon[34645]: pgmap v532: 97 pgs: 97 active+clean; 62 GiB data, 189 GiB used, 884 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 481 MiB/s wr, 2.38k op/s 2024-09-21T03:07:20.750 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.749+0000 7fb2135b6740 1 Processor -- start 2024-09-21T03:07:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.749+0000 7fb2135b6740 1 -- start start 2024-09-21T03:07:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d9d640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6db7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.751 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb208072ac0 con 0x5558c69ab9f0 2024-09-21T03:07:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb208075320 con 0x5558c69ed070 2024-09-21T03:07:20.752 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.750+0000 7fb2135b6740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb208073110 con 0x5558c69cc390 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.753+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.753+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:44544/0 (socket says 172.21.15.16:44544) 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.753+0000 7fb206d9d640 1 -- 172.21.15.16:0/1150615610 learned_addr learned my addr 172.21.15.16:0/1150615610 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.754+0000 7fb206d9d640 1 -- 172.21.15.16:0/1150615610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 msgr2=0x5558c6d9d640 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.754+0000 7fb206d9d640 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d9d640 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.754+0000 7fb206d9d640 1 -- 172.21.15.16:0/1150615610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 msgr2=0x5558c6db7c80 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.754+0000 7fb206d9d640 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6db7c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.754+0000 7fb206d9d640 1 -- 172.21.15.16:0/1150615610 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5558c6d92eb0 con 0x5558c69ab9f0 2024-09-21T03:07:20.760 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.756+0000 7fb206d9d640 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800bfc0 tx=0x7fb1f8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af5b3ef3e199cf98 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.758+0000 7fb1fcff9640 1 -- 172.21.15.16:0/1150615610 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f801e620 con 0x5558c69ab9f0 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.758+0000 7fb1fcff9640 1 -- 172.21.15.16:0/1150615610 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1f801ec70 con 0x5558c69ab9f0 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.758+0000 7fb1fcff9640 1 -- 172.21.15.16:0/1150615610 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f801dc80 con 0x5558c69ab9f0 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.759+0000 7fb2135b6740 1 -- 172.21.15.16:0/1150615610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 msgr2=0x5558c6d9d100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.759+0000 7fb2135b6740 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fb1f800bfc0 tx=0x7fb1f8013000 comp rx=0 tx=0).stop 2024-09-21T03:07:20.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 -- 172.21.15.16:0/1150615610 shutdown_connections 2024-09-21T03:07:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69ed070 0x5558c6db7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69cc390 0x5558c6d9d640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 --2- 172.21.15.16:0/1150615610 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ab9f0 0x5558c6d9d100 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.762 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 -- 172.21.15.16:0/1150615610 >> 172.21.15.16:0/1150615610 conn(0x5558c69e1460 msgr2=0x5558c6d9ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 -- 172.21.15.16:0/1150615610 shutdown_connections 2024-09-21T03:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.760+0000 7fb2135b6740 1 -- 172.21.15.16:0/1150615610 wait complete. 2024-09-21T03:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.761+0000 7fb2135b6740 1 -- start start 2024-09-21T03:07:20.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.762+0000 7fb2135b6740 1 Processor -- start 2024-09-21T03:07:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.762+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 0x5558c6da8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.763+0000 7fb2135b6740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 0x5558c6da3b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.763+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 0x5558c6da8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.763+0000 7fb206d9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 0x5558c6da8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:49554/0 (socket says 172.21.15.16:49554) 2024-09-21T03:07:20.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.763+0000 7fb206d9d640 1 -- 172.21.15.16:0/393102824 learned_addr learned my addr 172.21.15.16:0/393102824 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.763+0000 7fb1fffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 0x5558c6da3b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.764+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ed070 0x5558c6da4070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb2080737e0 con 0x5558c69ed070 2024-09-21T03:07:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb208074c50 con 0x5558c69cc390 2024-09-21T03:07:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb208073eb0 con 0x5558c69ab9f0 2024-09-21T03:07:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb2135b6740 10 client.0 _ll_register_callbacks cb 0x7fb1f000be30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:20.767 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb1fffff640 1 -- 172.21.15.16:0/393102824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 msgr2=0x5558c6da8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb1fffff640 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 0x5558c6da8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb1fffff640 1 -- 172.21.15.16:0/393102824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ed070 msgr2=0x5558c6da4070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb1fffff640 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ed070 0x5558c6da4070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.765+0000 7fb1fffff640 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5558c6da4630 con 0x5558c69cc390 2024-09-21T03:07:20.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.766+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1f801ec70 con 0x5558c69cc390 2024-09-21T03:07:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.766+0000 7fb1fffff640 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 0x5558c6da3b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5558c6c80c70 tx=0x7fb1f000b410 comp rx=0 tx=0).ready entity=mon.2 client_cookie=edf274e6d9ebc5fc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.767+0000 7fb1ee7fc640 10 client.25850 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:07:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.767+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/393102824 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb1f0031480 con 0x5558c69cc390 2024-09-21T03:07:20.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.768+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/393102824 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7fb1f0031620 con 0x5558c69cc390 2024-09-21T03:07:20.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.768+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/393102824 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb1f0005030 con 0x5558c69cc390 2024-09-21T03:07:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.774+0000 7fb2135b6740 10 client.25850 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:20.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.774+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=20}) -- 0x5558c6e1cae0 con 0x5558c69cc390 2024-09-21T03:07:20.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.780+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/393102824 <== mon.2 v2:172.21.15.18:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb1f00051d0 con 0x5558c69cc390 2024-09-21T03:07:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 10 client.25850 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 10 client.25850 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 20 client.25850 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6e1d070 0x5558c6e1f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:20.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 4 client.25850 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:07:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7fb1f801ec70 con 0x5558c6e1d070 2024-09-21T03:07:20.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.781+0000 7fb206d9d640 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6e1d070 0x5558c6e1f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:20.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.788+0000 7fb206d9d640 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6e1d070 0x5558c6e1f490 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8ca2474a3165d530 server_cookie=fe769f8041edba7 in_seq=0 out_seq=0 2024-09-21T03:07:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.788+0000 7fb1ee7fc640 10 client.25850 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.789+0000 7fb1ee7fc640 1 -- 172.21.15.16:0/393102824 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3978 (crc 0 0 0) 0x7fb1f801ec70 con 0x5558c6e1d070 2024-09-21T03:07:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.789+0000 7fb1ee7fc640 10 client.25850 handle_command_reply: tid=0 2024-09-21T03:07:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.789+0000 7fb2135b6740 1 client.25850 shutdown 2024-09-21T03:07:20.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.789+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 msgr2=0x5558c6da3b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.789+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 0x5558c6da3b30 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5558c6c80c70 tx=0x7fb1f000b410 comp rx=0 tx=0).stop 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 shutdown_connections 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5558c6e1d070 0x5558c6e1f490 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5558c69ed070 0x5558c6da4070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5558c69cc390 0x5558c6da3b30 secure :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0x5558c6c80c70 tx=0x7fb1f000b410 comp rx=0 tx=0).stop 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 --2- 172.21.15.16:0/393102824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5558c69ab9f0 0x5558c6da8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 >> 172.21.15.16:0/393102824 conn(0x5558c69e1460 msgr2=0x5558c6da13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.790+0000 7fb1ee7fc640 0 client.25850 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.791+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 shutdown_connections 2024-09-21T03:07:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.791+0000 7fb2135b6740 1 -- 172.21.15.16:0/393102824 wait complete. 2024-09-21T03:07:20.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.792+0000 7fb2135b6740 20 client.25850 trim_cache size 0 max 16384 2024-09-21T03:07:20.794 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "66226652-4934-4aed-bd6e-225c12fed8ba", 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:07:20.795 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.796 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.797 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.798 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.799 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.800 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:20.802 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:07:20.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 msgr2=0x7fb1f40b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 0x7fb1f40b8d60 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7fb208110620 tx=0x7fb1f800b410 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 msgr2=0x7fb208199a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208199a90 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fb200005e30 tx=0x7fb2000319f0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 shutdown_connections 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb1f40b6850 0x7fb1f40b8d60 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb208109e30 0x7fb208199a90 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb2081094e0 0x7fb208199550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 --2- 172.21.15.16:0/716990929 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb208103ba0 0x7fb208199010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.793+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 >> 172.21.15.16:0/716990929 conn(0x7fb2080ff630 msgr2=0x7fb208102b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.795+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 shutdown_connections 2024-09-21T03:07:20.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:20.797+0000 7fb20d042640 1 -- 172.21.15.16:0/716990929 wait complete. 2024-09-21T03:07:21.769 INFO:tasks.cephfs.filesystem:scrub status for tag:66226652-4934-4aed-bd6e-225c12fed8ba - {'path': '/', 'tag': '66226652-4934-4aed-bd6e-225c12fed8ba', 'options': 'recursive,force'} 2024-09-21T03:07:22.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:22 smithi018 ceph-mon[34587]: pgmap v533: 97 pgs: 97 active+clean; 63 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 2.44k op/s 2024-09-21T03:07:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:22 smithi123 ceph-mon[34645]: pgmap v533: 97 pgs: 97 active+clean; 63 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 2.44k op/s 2024-09-21T03:07:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:22 smithi016 ceph-mon[31253]: pgmap v533: 97 pgs: 97 active+clean; 63 GiB data, 190 GiB used, 882 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 462 MiB/s wr, 2.44k op/s 2024-09-21T03:07:23.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:23 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:07:24.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:23 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:07:24.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:23 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:07:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:24 smithi016 ceph-mon[31253]: pgmap v534: 97 pgs: 97 active+clean; 63 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 467 MiB/s wr, 2.41k op/s 2024-09-21T03:07:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:24 smithi018 ceph-mon[34587]: pgmap v534: 97 pgs: 97 active+clean; 63 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 467 MiB/s wr, 2.41k op/s 2024-09-21T03:07:25.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:24 smithi123 ceph-mon[34645]: pgmap v534: 97 pgs: 97 active+clean; 63 GiB data, 192 GiB used, 881 GiB / 1.0 TiB avail; 1.3 MiB/s rd, 467 MiB/s wr, 2.41k op/s 2024-09-21T03:07:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:26 smithi018 ceph-mon[34587]: pgmap v535: 97 pgs: 97 active+clean; 64 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 461 MiB/s wr, 1.93k op/s 2024-09-21T03:07:27.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:26 smithi123 ceph-mon[34645]: pgmap v535: 97 pgs: 97 active+clean; 64 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 461 MiB/s wr, 1.93k op/s 2024-09-21T03:07:27.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:26 smithi016 ceph-mon[31253]: pgmap v535: 97 pgs: 97 active+clean; 64 GiB data, 196 GiB used, 877 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 461 MiB/s wr, 1.93k op/s 2024-09-21T03:07:29.072 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:28 smithi016 ceph-mon[31253]: pgmap v536: 97 pgs: 97 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 445 MiB/s wr, 1.70k op/s 2024-09-21T03:07:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:28 smithi018 ceph-mon[34587]: pgmap v536: 97 pgs: 97 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 445 MiB/s wr, 1.70k op/s 2024-09-21T03:07:29.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:28 smithi123 ceph-mon[34645]: pgmap v536: 97 pgs: 97 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 445 MiB/s wr, 1.70k op/s 2024-09-21T03:07:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:30 smithi018 ceph-mon[34587]: pgmap v537: 97 pgs: 97 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 494 MiB/s wr, 1.79k op/s 2024-09-21T03:07:31.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:30 smithi123 ceph-mon[34645]: pgmap v537: 97 pgs: 97 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 494 MiB/s wr, 1.79k op/s 2024-09-21T03:07:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:30 smithi016 ceph-mon[31253]: pgmap v537: 97 pgs: 97 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 494 MiB/s wr, 1.79k op/s 2024-09-21T03:07:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:32 smithi018 ceph-mon[34587]: pgmap v538: 97 pgs: 97 active+clean; 68 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 1.94k op/s 2024-09-21T03:07:33.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:32 smithi123 ceph-mon[34645]: pgmap v538: 97 pgs: 97 active+clean; 68 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 1.94k op/s 2024-09-21T03:07:33.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:32 smithi016 ceph-mon[31253]: pgmap v538: 97 pgs: 97 active+clean; 68 GiB data, 208 GiB used, 865 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 635 MiB/s wr, 1.94k op/s 2024-09-21T03:07:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:34 smithi018 ceph-mon[34587]: pgmap v539: 97 pgs: 97 active+clean; 69 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 1.94k op/s 2024-09-21T03:07:35.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:34 smithi123 ceph-mon[34645]: pgmap v539: 97 pgs: 97 active+clean; 69 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 1.94k op/s 2024-09-21T03:07:35.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:34 smithi016 ceph-mon[31253]: pgmap v539: 97 pgs: 97 active+clean; 69 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 607 MiB/s wr, 1.94k op/s 2024-09-21T03:07:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:36 smithi018 ceph-mon[34587]: pgmap v540: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 634 MiB/s wr, 2.18k op/s 2024-09-21T03:07:37.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:36 smithi123 ceph-mon[34645]: pgmap v540: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 634 MiB/s wr, 2.18k op/s 2024-09-21T03:07:37.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:36 smithi016 ceph-mon[31253]: pgmap v540: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 634 MiB/s wr, 2.18k op/s 2024-09-21T03:07:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:38 smithi018 ceph-mon[34587]: pgmap v541: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 559 MiB/s wr, 1.91k op/s 2024-09-21T03:07:39.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:38 smithi123 ceph-mon[34645]: pgmap v541: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 559 MiB/s wr, 1.91k op/s 2024-09-21T03:07:39.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:38 smithi016 ceph-mon[31253]: pgmap v541: 97 pgs: 97 active+clean; 69 GiB data, 210 GiB used, 862 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 559 MiB/s wr, 1.91k op/s 2024-09-21T03:07:41.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:40 smithi016 ceph-mon[31253]: pgmap v542: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 505 MiB/s wr, 1.51k op/s 2024-09-21T03:07:41.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:40 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:41.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:40 smithi018 ceph-mon[34587]: pgmap v542: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 505 MiB/s wr, 1.51k op/s 2024-09-21T03:07:41.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:40 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:40 smithi123 ceph-mon[34645]: pgmap v542: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 505 MiB/s wr, 1.51k op/s 2024-09-21T03:07:41.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:40 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:07:42.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:42.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:41 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:42.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:42.184 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:41 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:07:42.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:07:42.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:41 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:07:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi018 to 177.0M 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Unable to set osd_memory_target on smithi123 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Unable to set osd_memory_target on smithi018 to 185699225: error parsing value: Value '185699225' is below minimum 939524096 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: pgmap v543: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 528 MiB/s wr, 1.23k op/s 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-21T03:07:43.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:42 smithi123 ceph-mon[34645]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi018 to 177.0M 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Unable to set osd_memory_target on smithi123 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Unable to set osd_memory_target on smithi018 to 185699225: error parsing value: Value '185699225' is below minimum 939524096 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: pgmap v543: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 528 MiB/s wr, 1.23k op/s 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-21T03:07:43.267 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:42 smithi016 ceph-mon[31253]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi123 to 176.0M 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi018 to 177.0M 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Unable to set osd_memory_target on smithi123 to 184589619: error parsing value: Value '184589619' is below minimum 939524096 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Unable to set osd_memory_target on smithi018 to 185699225: error parsing value: Value '185699225' is below minimum 939524096 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: pgmap v543: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 528 MiB/s wr, 1.23k op/s 2024-09-21T03:07:43.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Adjusting osd_memory_target on smithi016 to 177.0M 2024-09-21T03:07:43.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:42 smithi018 ceph-mon[34587]: Unable to set osd_memory_target on smithi016 to 185688473: error parsing value: Value '185688473' is below minimum 939524096 2024-09-21T03:07:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:45 smithi018 ceph-mon[34587]: pgmap v544: 97 pgs: 97 active+clean; 69 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 393 MiB/s wr, 803 op/s 2024-09-21T03:07:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:45 smithi123 ceph-mon[34645]: pgmap v544: 97 pgs: 97 active+clean; 69 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 393 MiB/s wr, 803 op/s 2024-09-21T03:07:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:45 smithi016 ceph-mon[31253]: pgmap v544: 97 pgs: 97 active+clean; 69 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 393 MiB/s wr, 803 op/s 2024-09-21T03:07:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:47 smithi018 ceph-mon[34587]: pgmap v545: 97 pgs: 97 active+clean; 70 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 472 MiB/s wr, 752 op/s 2024-09-21T03:07:47.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:47 smithi123 ceph-mon[34645]: pgmap v545: 97 pgs: 97 active+clean; 70 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 472 MiB/s wr, 752 op/s 2024-09-21T03:07:47.531 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:47 smithi016 ceph-mon[31253]: pgmap v545: 97 pgs: 97 active+clean; 70 GiB data, 213 GiB used, 860 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 472 MiB/s wr, 752 op/s 2024-09-21T03:07:49.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:49 smithi018 ceph-mon[34587]: pgmap v546: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 506 MiB/s wr, 523 op/s 2024-09-21T03:07:49.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:49 smithi123 ceph-mon[34645]: pgmap v546: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 506 MiB/s wr, 523 op/s 2024-09-21T03:07:49.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:49 smithi016 ceph-mon[31253]: pgmap v546: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 506 MiB/s wr, 523 op/s 2024-09-21T03:07:51.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:51 smithi016 ceph-mon[31253]: pgmap v547: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 518 MiB/s wr, 471 op/s 2024-09-21T03:07:51.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:51 smithi018 ceph-mon[34587]: pgmap v547: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 518 MiB/s wr, 471 op/s 2024-09-21T03:07:51.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:51 smithi123 ceph-mon[34645]: pgmap v547: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 2.4 MiB/s rd, 518 MiB/s wr, 471 op/s 2024-09-21T03:07:51.770 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:07:52.205 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:52.213 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:52 smithi016 ceph-mon[31253]: pgmap v548: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 593 MiB/s wr, 723 op/s 2024-09-21T03:07:52.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:52 smithi123 ceph-mon[34645]: pgmap v548: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 593 MiB/s wr, 723 op/s 2024-09-21T03:07:52.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:52 smithi018 ceph-mon[34587]: pgmap v548: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 593 MiB/s wr, 723 op/s 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/2400334822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0108d10 msgr2=0x7ff9d010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/2400334822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0108d10 0x7ff9d010f540 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8009900 tx=0x7ff9c802a410 comp rx=0 tx=0).stop 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/2400334822 shutdown_connections 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/2400334822 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0108d10 0x7ff9d010f540 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/2400334822 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d01087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/2400334822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0102ad0 0x7ff9d0102ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/2400334822 >> 172.21.15.16:0/2400334822 conn(0x7ff9d00796d0 msgr2=0x7ff9d0079ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.836+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/2400334822 shutdown_connections 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.837+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/2400334822 wait complete. 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.837+0000 7ff9d7ed9640 1 Processor -- start 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.837+0000 7ff9d7ed9640 1 -- start start 2024-09-21T03:07:52.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0102ad0 0x7ff9d019b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0108d10 0x7ff9d019bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01129d0 con 0x7ff9d0102ad0 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0112850 con 0x7ff9d01083c0 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d7ed9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d0112b50 con 0x7ff9d0108d10 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:45450/0 (socket says 172.21.15.16:45450) 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 -- 172.21.15.16:0/470702435 learned_addr learned my addr 172.21.15.16:0/470702435 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0102ad0 0x7ff9d019b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d6476640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0108d10 0x7ff9d019bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 -- 172.21.15.16:0/470702435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0108d10 msgr2=0x7ff9d019bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0108d10 0x7ff9d019bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 -- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0102ad0 msgr2=0x7ff9d019b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.838+0000 7ff9d5474640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0102ad0 0x7ff9d019b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9d5474640 1 -- 172.21.15.16:0/470702435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9d01a6540 con 0x7ff9d01083c0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9d5474640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d950 tx=0x7ff9c000de10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9c6ffd640 1 -- 172.21.15.16:0/470702435 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c0017070 con 0x7ff9d01083c0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9c6ffd640 1 -- 172.21.15.16:0/470702435 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0009c30 con 0x7ff9d01083c0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9c6ffd640 1 -- 172.21.15.16:0/470702435 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00047a0 con 0x7ff9d01083c0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9d01990d0 con 0x7ff9d01083c0 2024-09-21T03:07:52.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.839+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9d0199500 con 0x7ff9d01083c0 2024-09-21T03:07:52.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.842+0000 7ff9c6ffd640 1 -- 172.21.15.16:0/470702435 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff9c0004020 con 0x7ff9d01083c0 2024-09-21T03:07:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.843+0000 7ff9c6ffd640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 0x7ff9bc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 Processor -- start 2024-09-21T03:07:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9c6ffd640 1 -- 172.21.15.16:0/470702435 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7ff9c00e7270 con 0x7ff9d01083c0 2024-09-21T03:07:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 0x7ff9bc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 -- start start 2024-09-21T03:07:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6be5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 0x7ff9bc0b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff9d006c6c0 tx=0x7ff9cc012040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fb6b5be50 con 0x562fb681d390 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562fb6b4b280 con 0x562fb683e070 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9de44d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562fb6b4f090 con 0x562fb68151b0 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.844+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6be5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:45458/0 (socket says 172.21.15.16:45458) 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d5c75640 1 -- 172.21.15.16:0/4000532918 learned_addr learned my addr 172.21.15.16:0/4000532918 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d5474640 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 -- 172.21.15.16:0/4000532918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 msgr2=0x562fb6be57b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be57b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 -- 172.21.15.16:0/4000532918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 msgr2=0x562fb6be5270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d6476640 1 -- 172.21.15.16:0/4000532918 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb68a9650 con 0x562fb681d390 2024-09-21T03:07:52.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.845+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.846+0000 7ff9d6476640 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6be5cf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8005940 tx=0x7ff9c8032bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=494b0ef5185b0399 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.846+0000 7ff9bbfff640 1 -- 172.21.15.16:0/4000532918 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8040070 con 0x562fb681d390 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.846+0000 7ff9bbfff640 1 -- 172.21.15.16:0/4000532918 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c8004030 con 0x562fb681d390 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.846+0000 7ff9bbfff640 1 -- 172.21.15.16:0/4000532918 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c8035040 con 0x562fb681d390 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 -- 172.21.15.16:0/4000532918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 msgr2=0x562fb6be5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6be5cf0 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7ff9c8005940 tx=0x7ff9c8032bd0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 -- 172.21.15.16:0/4000532918 shutdown_connections 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6be5cf0 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be57b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 --2- 172.21.15.16:0/4000532918 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6be5270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 -- 172.21.15.16:0/4000532918 >> 172.21.15.16:0/4000532918 conn(0x562fb6832460 msgr2=0x562fb6be3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 -- 172.21.15.16:0/4000532918 shutdown_connections 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.847+0000 7ff9de44d740 1 -- 172.21.15.16:0/4000532918 wait complete. 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9de44d740 1 -- start start 2024-09-21T03:07:52.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9de44d740 1 Processor -- start 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6bf9170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 0x562fb6bf96b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 0x562fb6c05450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.848+0000 7ff9d5474640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 0x562fb6bf96b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6bf9170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9de44d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fb69ac700 con 0x562fb683e070 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9de44d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562fb6ae6110 con 0x562fb681d390 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9de44d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562fb6bfc120 con 0x562fb68151b0 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6bf9170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:43918/0 (socket says 172.21.15.16:43918) 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5c75640 1 -- 172.21.15.16:0/3081220616 learned_addr learned my addr 172.21.15.16:0/3081220616 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d6476640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 0x562fb6c05450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9de44d740 10 client.0 _ll_register_callbacks cb 0x562fb6aff990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 -- 172.21.15.16:0/3081220616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 msgr2=0x562fb6bf9170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6bf9170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 -- 172.21.15.16:0/3081220616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 msgr2=0x562fb6c05450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 0x562fb6c05450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 -- 172.21.15.16:0/3081220616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb6b13c40 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d6476640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 0x562fb6c05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 0x562fb6bf96b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00e49b0 tx=0x7ff9c00acfb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7d8fdb52e9000bd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.849+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00a8c00 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7ff9c00abd80 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c8004030 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff9c00acbc0 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 10 client.25862 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562fb6c07ee0 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c00ad7b0 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7ff9c00abf80 con 0x562fb681d390 2024-09-21T03:07:52.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 10 client.25862 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 10 client.25862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 20 client.25862 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67fc9f0 0x562fb6c0b350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 4 client.25862 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7ff9c8004030 con 0x562fb67fc9f0 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.850+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67fc9f0 0x562fb6c0b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.851+0000 7ff9b97fa640 10 client.25862 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:52.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.851+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67fc9f0 0x562fb6c0b350 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b582ce98679dc4a server_cookie=3303ab711fcb194a in_seq=0 out_seq=0 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3081220616 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7ff9c8004030 con 0x562fb67fc9f0 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9b97fa640 10 client.25862 handle_command_reply: tid=0 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 client.25862 shutdown 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 msgr2=0x562fb6bf96b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 0x562fb6bf96b0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7ff9c00e49b0 tx=0x7ff9c00acfb0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 shutdown_connections 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67fc9f0 0x562fb6c0b350 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb683e070 0x562fb6c05450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9b97fa640 0 client.25862 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb681d390 0x562fb6bf96b0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3081220616 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6bf9170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 >> 172.21.15.16:0/3081220616 conn(0x562fb6832460 msgr2=0x562fb6be1700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 shutdown_connections 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 1 -- 172.21.15.16:0/3081220616 wait complete. 2024-09-21T03:07:52.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.852+0000 7ff9de44d740 20 client.25862 trim_cache size 0 max 16384 2024-09-21T03:07:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 Processor -- start 2024-09-21T03:07:52.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 -- start start 2024-09-21T03:07:52.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6bf9460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ab580 con 0x562fb681d390 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01adde0 con 0x562fb683e070 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.870+0000 7ff9de44d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01abbd0 con 0x562fb68151b0 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:52132/0 (socket says 172.21.15.16:52132) 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 -- 172.21.15.16:0/3711247402 learned_addr learned my addr 172.21.15.16:0/3711247402 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d6476640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6bf9460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 -- 172.21.15.16:0/3711247402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 msgr2=0x562fb6be2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be2240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 -- 172.21.15.16:0/3711247402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 msgr2=0x562fb6bf9460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6bf9460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 -- 172.21.15.16:0/3711247402 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fb6be8080 con 0x562fb681d390 2024-09-21T03:07:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d6476640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6bf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:52.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.871+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005fd0 tx=0x7ff9c0005190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce307fceeb6f42d3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.872+0000 7ff9bb7fe640 1 -- 172.21.15.16:0/3711247402 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00ad7b0 con 0x562fb681d390 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.872+0000 7ff9bb7fe640 1 -- 172.21.15.16:0/3711247402 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0009de0 con 0x562fb681d390 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.872+0000 7ff9bb7fe640 1 -- 172.21.15.16:0/3711247402 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9c00abd80 con 0x562fb681d390 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- 172.21.15.16:0/3711247402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 msgr2=0x562fb6bf8f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7ff9c0005fd0 tx=0x7ff9c0005190 comp rx=0 tx=0).stop 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- 172.21.15.16:0/3711247402 shutdown_connections 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6bf8f20 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb683e070 0x562fb6bf9460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3711247402 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb68151b0 0x562fb6be2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- 172.21.15.16:0/3711247402 >> 172.21.15.16:0/3711247402 conn(0x562fb6832460 msgr2=0x562fb6bf79f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- 172.21.15.16:0/3711247402 shutdown_connections 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- 172.21.15.16:0/3711247402 wait complete. 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 -- start start 2024-09-21T03:07:52.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.873+0000 7ff9de44d740 1 Processor -- start 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6c01900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb683e070 0x562fb6bfc370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ac2a0 con 0x562fb681d390 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ad710 con 0x562fb68151b0 2024-09-21T03:07:52.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff9d01ac970 con 0x562fb683e070 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9de44d740 10 client.0 _ll_register_callbacks cb 0x7ff9c000bf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9d6476640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb683e070 0x562fb6bfc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9d5c75640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:45490/0 (socket says 172.21.15.16:45490) 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9d5c75640 1 -- 172.21.15.16:0/3404966913 learned_addr learned my addr 172.21.15.16:0/3404966913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.874+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6c01900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 -- 172.21.15.16:0/3404966913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb683e070 msgr2=0x562fb6bfc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb683e070 0x562fb6bfc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 -- 172.21.15.16:0/3404966913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 msgr2=0x562fb6c01900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6c01900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 -- 172.21.15.16:0/3404966913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fb6c02040 con 0x562fb68151b0 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6c01900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:52.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9d5c75640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562fb6c0ba80 tx=0x7ff9cc004a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=16ea477002474f88 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9b97fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc010670 con 0x562fb68151b0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7ff9cc010810 con 0x562fb68151b0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.875+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9c0009de0 con 0x562fb68151b0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff9cc010a10 con 0x562fb68151b0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 10 client.25868 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562fb6c76230 con 0x562fb68151b0 2024-09-21T03:07:52.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9c0009de0 con 0x562fb68151b0 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7ff9cc01e410 con 0x562fb68151b0 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 10 client.25868 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 10 client.25868 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 20 client.25868 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67d1da0 0x562fb6c789d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 4 client.25868 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.876+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x562fb6b5dda0 con 0x562fb67d1da0 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.877+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67d1da0 0x562fb6c789d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.877+0000 7ff9b97fa640 10 client.25868 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:52.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.877+0000 7ff9d5474640 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67d1da0 0x562fb6c789d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7edc0c45c5e5289b server_cookie=6a56e05731eb1378 in_seq=0 out_seq=0 2024-09-21T03:07:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9b97fa640 1 -- 172.21.15.16:0/3404966913 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3782 (crc 0 0 0) 0x562fb6b5dda0 con 0x562fb67d1da0 2024-09-21T03:07:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9b97fa640 10 client.25868 handle_command_reply: tid=0 2024-09-21T03:07:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 client.25868 shutdown 2024-09-21T03:07:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 msgr2=0x562fb6c013c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x562fb6c0ba80 tx=0x7ff9cc004a40 comp rx=0 tx=0).stop 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 shutdown_connections 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fb67d1da0 0x562fb6c789d0 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fb681d390 0x562fb6c01900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fb683e070 0x562fb6bfc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9b97fa640 0 client.25868 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 --2- 172.21.15.16:0/3404966913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fb68151b0 0x562fb6c013c0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.879+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 >> 172.21.15.16:0/3404966913 conn(0x562fb6832460 msgr2=0x562fb6be21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 shutdown_connections 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9de44d740 1 -- 172.21.15.16:0/3404966913 wait complete. 2024-09-21T03:07:52.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9de44d740 20 client.25868 trim_cache size 0 max 16384 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:07:52.883 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:07:52.884 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:07:52.885 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:07:52.886 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:07:52.887 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:07:52.889 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 msgr2=0x7ff9bc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 0x7ff9bc0b8c10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ff9d006c6c0 tx=0x7ff9cc012040 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 msgr2=0x7ff9d019b7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7ff9c000d950 tx=0x7ff9c000de10 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 shutdown_connections 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff9bc0b6700 0x7ff9bc0b8c10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.880+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff9d0108d10 0x7ff9d019bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.881+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff9d01083c0 0x7ff9d019b7c0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.881+0000 7ff9d7ed9640 1 --2- 172.21.15.16:0/470702435 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff9d0102ad0 0x7ff9d019b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:52.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.881+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 >> 172.21.15.16:0/470702435 conn(0x7ff9d00796d0 msgr2=0x7ff9d0106450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.881+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 shutdown_connections 2024-09-21T03:07:52.891 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:52.881+0000 7ff9d7ed9640 1 -- 172.21.15.16:0/470702435 wait complete. 2024-09-21T03:07:53.768 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:07:53.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:07:54.180 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:54.654 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:54 smithi016 ceph-mon[31253]: pgmap v549: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 514 MiB/s wr, 745 op/s 2024-09-21T03:07:54.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:54 smithi018 ceph-mon[34587]: pgmap v549: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 514 MiB/s wr, 745 op/s 2024-09-21T03:07:54.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:54 smithi123 ceph-mon[34645]: pgmap v549: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 514 MiB/s wr, 745 op/s 2024-09-21T03:07:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.988+0000 7f76b60bc640 1 -- 172.21.15.16:0/1887183088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 msgr2=0x7f76b0112840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:54.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.989+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1887183088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b0112840 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f76a8009ca0 tx=0x7f76a802a8d0 comp rx=0 tx=0).stop 2024-09-21T03:07:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.990+0000 7f76b60bc640 1 -- 172.21.15.16:0/1887183088 shutdown_connections 2024-09-21T03:07:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.990+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1887183088 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b0112840 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.990+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1887183088 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b010b6c0 0x7f76b010bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.990+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1887183088 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76b0105d80 0x7f76b0106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:54.992 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.990+0000 7f76b60bc640 1 -- 172.21.15.16:0/1887183088 >> 172.21.15.16:0/1887183088 conn(0x7f76b01017f0 msgr2=0x7f76b0103c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:54.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.992+0000 7f76b60bc640 1 -- 172.21.15.16:0/1887183088 shutdown_connections 2024-09-21T03:07:54.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.993+0000 7f76b60bc640 1 -- 172.21.15.16:0/1887183088 wait complete. 2024-09-21T03:07:54.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.994+0000 7f76b60bc640 1 Processor -- start 2024-09-21T03:07:54.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.995+0000 7f76b60bc640 1 -- start start 2024-09-21T03:07:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.995+0000 7f76b60bc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76b0105d80 0x7f76b019c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:54.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76b60bc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b010b6c0 0x7f76b019ca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:54.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76b60bc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76b60bc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0115d30 con 0x7f76b010c010 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76b60bc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0115bb0 con 0x7f76b010b6c0 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76b60bc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f76b0115eb0 con 0x7f76b0105d80 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76affff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49712/0 (socket says 172.21.15.16:49712) 2024-09-21T03:07:54.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.996+0000 7f76affff640 1 -- 172.21.15.16:0/1474086311 learned_addr learned my addr 172.21.15.16:0/1474086311 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.997+0000 7f76affff640 1 -- 172.21.15.16:0/1474086311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76b0105d80 msgr2=0x7f76b019c4f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.997+0000 7f76affff640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76b0105d80 0x7f76b019c4f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.997+0000 7f76affff640 1 -- 172.21.15.16:0/1474086311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b010b6c0 msgr2=0x7f76b019ca90 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.997+0000 7f76affff640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b010b6c0 0x7f76b019ca90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.997+0000 7f76affff640 1 -- 172.21.15.16:0/1474086311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76b01a2460 con 0x7f76b010c010 2024-09-21T03:07:55.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.998+0000 7f76affff640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f76a8003ba0 tx=0x7f76a8003bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.999+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8004040 con 0x7f76b010c010 2024-09-21T03:07:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.999+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76a8004690 con 0x7f76b010c010 2024-09-21T03:07:55.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.999+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76b01a26f0 con 0x7f76b010c010 2024-09-21T03:07:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:54.999+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76a8030d50 con 0x7f76b010c010 2024-09-21T03:07:55.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.000+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76b01a8eb0 con 0x7f76b010c010 2024-09-21T03:07:55.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.003+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76b019d620 con 0x7f76b010c010 2024-09-21T03:07:55.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.006+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f76a80041e0 con 0x7f76b010c010 2024-09-21T03:07:55.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.009+0000 7f76acff9640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 0x7f76980b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:55.011 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.009+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f76a8108b20 con 0x7f76b010c010 2024-09-21T03:07:55.012 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.010+0000 7f76af7fe640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 0x7f76980b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:55.013 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.011+0000 7f76af7fe640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 0x7f76980b8cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f76a00096b0 tx=0x7f76a0006c20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:55.017 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.015+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f76a80cf000 con 0x7f76b010c010 2024-09-21T03:07:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.302+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f76b01a93f0 con 0x7f76b010c010 2024-09-21T03:07:55.304 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.303+0000 7f76acff9640 1 -- 172.21.15.16:0/1474086311 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f76a80d33e0 con 0x7f76b010c010 2024-09-21T03:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:07:55.305 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:07:55.305 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:07:55.308 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 msgr2=0x7f76980b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 0x7f76980b8cc0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f76a00096b0 tx=0x7f76a0006c20 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 msgr2=0x7f76b01a1d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f76a8003ba0 tx=0x7f76a8003bd0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 shutdown_connections 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f76980b67b0 0x7f76980b8cc0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f76b010c010 0x7f76b01a1d30 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f76b010b6c0 0x7f76b019ca90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 --2- 172.21.15.16:0/1474086311 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f76b0105d80 0x7f76b019c4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 >> 172.21.15.16:0/1474086311 conn(0x7f76b01017f0 msgr2=0x7f76b01094d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 shutdown_connections 2024-09-21T03:07:55.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:55.307+0000 7f76b60bc640 1 -- 172.21.15.16:0/1474086311 wait complete. 2024-09-21T03:07:55.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:55 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1474086311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:55.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:55 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1474086311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:56.011 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:55 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1474086311' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:07:56.098 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:07:56.418 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:07:56.857 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:56 smithi016 ceph-mon[31253]: pgmap v550: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 535 MiB/s wr, 830 op/s 2024-09-21T03:07:56.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:56 smithi018 ceph-mon[34587]: pgmap v550: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 535 MiB/s wr, 830 op/s 2024-09-21T03:07:56.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:56 smithi123 ceph-mon[34645]: pgmap v550: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 535 MiB/s wr, 830 op/s 2024-09-21T03:07:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 -- 172.21.15.16:0/3717513031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281083b0 msgr2=0x7f43281087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 --2- 172.21.15.16:0/3717513031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281083b0 0x7f43281087c0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f4310009900 tx=0x7f431002a410 comp rx=0 tx=0).stop 2024-09-21T03:07:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 -- 172.21.15.16:0/3717513031 shutdown_connections 2024-09-21T03:07:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 --2- 172.21.15.16:0/3717513031 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f43281083b0 0x7f43281087c0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 --2- 172.21.15.16:0/3717513031 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4328108d00 0x7f432810f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 --2- 172.21.15.16:0/3717513031 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328102a70 0x7f4328102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 -- 172.21.15.16:0/3717513031 >> 172.21.15.16:0/3717513031 conn(0x7f4328079730 msgr2=0x7f4328079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 -- 172.21.15.16:0/3717513031 shutdown_connections 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.123+0000 7f432dc1e640 1 -- 172.21.15.16:0/3717513031 wait complete. 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.124+0000 7f432dc1e640 1 Processor -- start 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.124+0000 7f432dc1e640 1 -- start start 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 0x7f432810ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328108d00 0x7f4328110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4328113a90 con 0x7f4328102a70 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4328113910 con 0x7f43281083b0 2024-09-21T03:07:57.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f432dc1e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4328113c10 con 0x7f4328108d00 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53074/0 (socket says 172.21.15.16:53074) 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 -- 172.21.15.16:0/566403735 learned_addr learned my addr 172.21.15.16:0/566403735 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4327fff640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328108d00 0x7f4328110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f43277fe640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 0x7f432810ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 -- 172.21.15.16:0/566403735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328108d00 msgr2=0x7f4328110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328108d00 0x7f4328110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 -- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 msgr2=0x7f432810ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 0x7f432810ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f4326ffd640 1 -- 172.21.15.16:0/566403735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43281a8760 con 0x7f43281083b0 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.125+0000 7f43277fe640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 0x7f432810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:07:57.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f4326ffd640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4310006010 tx=0x7f4310004060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f4324ff9640 1 -- 172.21.15.16:0/566403735 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431003f030 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43281a89f0 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f4324ff9640 1 -- 172.21.15.16:0/566403735 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4310004490 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f4324ff9640 1 -- 172.21.15.16:0/566403735 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4310035070 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.126+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43281a8ed0 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.128+0000 7f4334192740 1 Processor -- start 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.128+0000 7f4334192740 1 -- start start 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4324ff9640 1 -- 172.21.15.16:0/566403735 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4310042030 con 0x7f43281083b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2db31b0 0x5622f3183800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2dbb390 0x5622f3183d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622f30f9ea0 con 0x5622f2dbb390 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5622f30e92d0 con 0x5622f2db31b0 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4334192740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622f30ed0e0 con 0x5622f2ddc070 2024-09-21T03:07:57.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f43277fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2db31b0 0x5622f3183800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2dbb390 0x5622f3183d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47390/0 (socket says 172.21.15.16:47390) 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 -- 172.21.15.16:0/1676612645 learned_addr learned my addr 172.21.15.16:0/1676612645 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 -- 172.21.15.16:0/1676612645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2db31b0 msgr2=0x5622f3183800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2db31b0 0x5622f3183800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 -- 172.21.15.16:0/1676612645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2dbb390 msgr2=0x5622f3183d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2dbb390 0x5622f3183d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 -- 172.21.15.16:0/1676612645 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f2e47650 con 0x5622f2ddc070 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.129+0000 7f4327fff640 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4320011440 tx=0x7f4320011900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e188ceec3376a9c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f4324ff9640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 0x7f43140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f431dffb640 1 -- 172.21.15.16:0/1676612645 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320003c30 con 0x5622f2ddc070 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f4324ff9640 1 -- 172.21.15.16:0/566403735 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f4310107be0 con 0x7f43281083b0 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f431dffb640 1 -- 172.21.15.16:0/1676612645 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320003dd0 con 0x5622f2ddc070 2024-09-21T03:07:57.136 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f431dffb640 1 -- 172.21.15.16:0/1676612645 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000fae0 con 0x5622f2ddc070 2024-09-21T03:07:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.130+0000 7f43277fe640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 0x7f43140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- 172.21.15.16:0/1676612645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 msgr2=0x5622f31832c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f43277fe640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 0x7f43140b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5622f3181f40 tx=0x7f43180094c0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f4320011440 tx=0x7f4320011900 comp rx=0 tx=0).stop 2024-09-21T03:07:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- 172.21.15.16:0/1676612645 shutdown_connections 2024-09-21T03:07:57.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2dbb390 0x5622f3183d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2db31b0 0x5622f3183800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 --2- 172.21.15.16:0/1676612645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2ddc070 0x5622f31832c0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- 172.21.15.16:0/1676612645 >> 172.21.15.16:0/1676612645 conn(0x5622f2dd0460 msgr2=0x5622f31811b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- 172.21.15.16:0/1676612645 shutdown_connections 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- 172.21.15.16:0/1676612645 wait complete. 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.131+0000 7f4334192740 1 -- start start 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 Processor -- start 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2db31b0 0x5622f318ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5622f2f4a700 con 0x5622f2db31b0 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5622f3084160 con 0x5622f2ddc070 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5622f3191aa0 con 0x5622f2dbb390 2024-09-21T03:07:57.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4334192740 10 client.0 _ll_register_callbacks cb 0x5622f309d9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f43277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2db31b0 0x5622f318ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.132+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47400/0 (socket says 172.21.15.16:47400) 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53094/0 (socket says 172.21.15.16:53094) 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4326ffd640 1 -- 172.21.15.16:0/486448826 learned_addr learned my addr 172.21.15.16:0/486448826 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 -- 172.21.15.16:0/486448826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 msgr2=0x5622f318ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 -- 172.21.15.16:0/486448826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2db31b0 msgr2=0x5622f318ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2db31b0 0x5622f318ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 -- 172.21.15.16:0/486448826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f30b1c90 con 0x5622f2ddc070 2024-09-21T03:07:57.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4326ffd640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4327fff640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f432001b040 tx=0x7f4320004c70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94fc1a131239a6db server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4306ffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320028070 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f432000fa30 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.133+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4320003dd0 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4320034050 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 10 client.25883 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5622f319d860 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320003dd0 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f43200045d0 con 0x5622f2ddc070 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 10 client.25883 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 10 client.25883 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 20 client.25883 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2d9a9f0 0x5622f31a0c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 4 client.25883 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5622f30fbdf0 con 0x5622f2d9a9f0 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.134+0000 7f43277fe640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2d9a9f0 0x5622f31a0c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.135+0000 7f4306ffd640 10 client.25883 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.135+0000 7f43277fe640 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2d9a9f0 0x5622f31a0c20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a139e5323b078905 server_cookie=8a46dcd4bb7eec4e in_seq=0 out_seq=0 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4306ffd640 1 -- 172.21.15.16:0/486448826 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5622f30fbdf0 con 0x5622f2d9a9f0 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4306ffd640 10 client.25883 handle_command_reply: tid=0 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 client.25883 shutdown 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 msgr2=0x5622f319adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f432001b040 tx=0x7f4320004c70 comp rx=0 tx=0).stop 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 shutdown_connections 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2d9a9f0 0x5622f31a0c20 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4306ffd640 0 client.25883 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2ddc070 0x5622f319adc0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2dbb390 0x5622f318ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 --2- 172.21.15.16:0/486448826 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2db31b0 0x5622f318ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.136+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 >> 172.21.15.16:0/486448826 conn(0x5622f2dd0460 msgr2=0x5622f3186450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.137+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 shutdown_connections 2024-09-21T03:07:57.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.137+0000 7f4334192740 1 -- 172.21.15.16:0/486448826 wait complete. 2024-09-21T03:07:57.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.137+0000 7f4334192740 20 client.25883 trim_cache size 0 max 16384 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 Processor -- start 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 -- start start 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3186ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f319acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ab580 con 0x5622f2d9a9f0 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43281addc0 con 0x5622f2dbb390 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4334192740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43281abbd0 con 0x5622f2db31b0 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f43277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3186ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4327fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4327fff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53108/0 (socket says 172.21.15.16:53108) 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f4326ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f319acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 -- 172.21.15.16:0/2315538135 learned_addr learned my addr 172.21.15.16:0/2315538135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:57.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.151+0000 7f43277fe640 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3186ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49742/0 (socket says 172.21.15.16:49742) 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 -- 172.21.15.16:0/2315538135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 msgr2=0x5622f319acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f319acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 -- 172.21.15.16:0/2315538135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 msgr2=0x5622f3186ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3186ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 -- 172.21.15.16:0/2315538135 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622f318cf20 con 0x5622f2dbb390 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4326ffd640 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f319acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:57.155 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f4327fff640 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f432000f450 tx=0x7f4320005cf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8651aabc1dfbe3aa server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f431d7fa640 1 -- 172.21.15.16:0/2315538135 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4320028040 con 0x5622f2dbb390 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f431d7fa640 1 -- 172.21.15.16:0/2315538135 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320003d80 con 0x5622f2dbb390 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.152+0000 7f431d7fa640 1 -- 172.21.15.16:0/2315538135 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f432000fa30 con 0x5622f2dbb390 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 -- 172.21.15.16:0/2315538135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 msgr2=0x5622f318e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f432000f450 tx=0x7f4320005cf0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 -- 172.21.15.16:0/2315538135 shutdown_connections 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f318e7d0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f319acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 --2- 172.21.15.16:0/2315538135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3186ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 -- 172.21.15.16:0/2315538135 >> 172.21.15.16:0/2315538135 conn(0x5622f2dd0460 msgr2=0x5622f31863c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 -- 172.21.15.16:0/2315538135 shutdown_connections 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.153+0000 7f4334192740 1 -- 172.21.15.16:0/2315538135 wait complete. 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 -- start start 2024-09-21T03:07:57.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 Processor -- start 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f3192930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f3192e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ac280 con 0x5622f2d9a9f0 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ad6f0 con 0x5622f2dbb390 2024-09-21T03:07:57.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f43281ac950 con 0x5622f2db31b0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4334192740 10 client.0 _ll_register_callbacks cb 0x7f431010b600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f43277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f43277fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49750/0 (socket says 172.21.15.16:49750) 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f43277fe640 1 -- 172.21.15.16:0/611486046 learned_addr learned my addr 172.21.15.16:0/611486046 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4326ffd640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f3192930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.154+0000 7f4327fff640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f3192e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 -- 172.21.15.16:0/611486046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 msgr2=0x5622f3192930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f3192930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 -- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 msgr2=0x5622f3192e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f3192e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 -- 172.21.15.16:0/611486046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622f31935b0 con 0x5622f2d9a9f0 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4326ffd640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f3192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4327fff640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f3192e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:07:57.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f43277fe640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4318005fd0 tx=0x7f43180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa448ea16cd1b208 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4306ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f431803b060 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f431801d030 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4320003d80 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f4318024870 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4334192740 10 client.16230 fetch_fsmap learned FSMap version 20 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.155+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5622f320b3e0 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4320003d80 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f431800f040 con 0x5622f2d9a9f0 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 10 client.16230 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 10 client.16230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 20 client.16230 populate_metadata read hostname 'smithi016' 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2ddc070 0x5622f320dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 4 client.16230 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5622f30f78a0 con 0x5622f2ddc070 2024-09-21T03:07:57.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4326ffd640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2ddc070 0x5622f320dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4306ffd640 10 client.16230 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.156+0000 7f4326ffd640 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2ddc070 0x5622f320dbb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4aff7425e6485883 server_cookie=d955e4bd46e486b3 in_seq=0 out_seq=0 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4306ffd640 1 -- 172.21.15.16:0/611486046 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5622f30f78a0 con 0x5622f2ddc070 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4306ffd640 10 client.16230 handle_command_reply: tid=0 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 client.16230 shutdown 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 msgr2=0x5622f3197480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f4318005fd0 tx=0x7f43180046c0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 shutdown_connections 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5622f2ddc070 0x5622f320dbb0 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4306ffd640 0 client.16230 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5622f2dbb390 0x5622f3192e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5622f2db31b0 0x5622f3192930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 --2- 172.21.15.16:0/611486046 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5622f2d9a9f0 0x5622f3197480 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 >> 172.21.15.16:0/611486046 conn(0x5622f2dd0460 msgr2=0x5622f318fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 shutdown_connections 2024-09-21T03:07:57.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 1 -- 172.21.15.16:0/611486046 wait complete. 2024-09-21T03:07:57.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.157+0000 7f4334192740 20 client.16230 trim_cache size 0 max 16384 2024-09-21T03:07:57.163 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 msgr2=0x7f43140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 0x7f43140b8b90 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x5622f3181f40 tx=0x7f43180094c0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 msgr2=0x7f4328110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f4310006010 tx=0x7f4310004060 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 shutdown_connections 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f43140b6680 0x7f43140b8b90 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4328108d00 0x7f4328110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f43281083b0 0x7f4328110500 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 --2- 172.21.15.16:0/566403735 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4328102a70 0x7f432810ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 >> 172.21.15.16:0/566403735 conn(0x7f4328079730 msgr2=0x7f4328104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 shutdown_connections 2024-09-21T03:07:57.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:07:57.158+0000 7f432dc1e640 1 -- 172.21.15.16:0/566403735 wait complete. 2024-09-21T03:07:58.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:07:58 smithi018 ceph-mon[34587]: pgmap v551: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 456 MiB/s wr, 821 op/s 2024-09-21T03:07:58.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:07:58 smithi123 ceph-mon[34645]: pgmap v551: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 456 MiB/s wr, 821 op/s 2024-09-21T03:07:59.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:07:58 smithi016 ceph-mon[31253]: pgmap v551: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 456 MiB/s wr, 821 op/s 2024-09-21T03:07:59.128 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:07:59.128 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:07:59.427 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.144+0000 7f657efc3640 1 -- 172.21.15.16:0/3865330399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578114600 msgr2=0x7f6578116a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.144+0000 7f657efc3640 1 --2- 172.21.15.16:0/3865330399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578114600 0x7f6578116a20 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6570009ca0 tx=0x7f657002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 -- 172.21.15.16:0/3865330399 shutdown_connections 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 --2- 172.21.15.16:0/3865330399 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578114600 0x7f6578116a20 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 --2- 172.21.15.16:0/3865330399 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578111ca0 0x7f65781140c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 --2- 172.21.15.16:0/3865330399 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578101ff0 0x7f6578102400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 -- 172.21.15.16:0/3865330399 >> 172.21.15.16:0/3865330399 conn(0x7f6578077cc0 msgr2=0x7f6578078100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.146+0000 7f657efc3640 1 -- 172.21.15.16:0/3865330399 shutdown_connections 2024-09-21T03:08:00.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.147+0000 7f657efc3640 1 -- 172.21.15.16:0/3865330399 wait complete. 2024-09-21T03:08:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 Processor -- start 2024-09-21T03:08:00.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 -- start start 2024-09-21T03:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578101ff0 0x7f65781a0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 0x7f65781a1030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f657811a0e0 con 0x7f6578114600 2024-09-21T03:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6578119f60 con 0x7f6578111ca0 2024-09-21T03:08:00.152 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657efc3640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f657811a260 con 0x7f6578101ff0 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.148+0000 7f657d560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.149+0000 7f657d560640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49766/0 (socket says 172.21.15.16:49766) 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.149+0000 7f657d560640 1 -- 172.21.15.16:0/461693526 learned_addr learned my addr 172.21.15.16:0/461693526 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.149+0000 7f657cd5f640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578101ff0 0x7f65781a0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.149+0000 7f6577fff640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 0x7f65781a1030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 -- 172.21.15.16:0/461693526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578101ff0 msgr2=0x7f65781a0a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578101ff0 0x7f65781a0a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 -- 172.21.15.16:0/461693526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 msgr2=0x7f65781a1030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 0x7f65781a1030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 -- 172.21.15.16:0/461693526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f65781a6a00 con 0x7f6578114600 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f6577fff640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 0x7f65781a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.150+0000 7f657d560640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6570005cc0 tx=0x7f657002f380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.151+0000 7f6575ffb640 1 -- 172.21.15.16:0/461693526 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6570003bf0 con 0x7f6578114600 2024-09-21T03:08:00.153 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.151+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65781a6c90 con 0x7f6578114600 2024-09-21T03:08:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.151+0000 7f6575ffb640 1 -- 172.21.15.16:0/461693526 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6570003d90 con 0x7f6578114600 2024-09-21T03:08:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.151+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65781ad450 con 0x7f6578114600 2024-09-21T03:08:00.154 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.151+0000 7f6575ffb640 1 -- 172.21.15.16:0/461693526 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f657002fbc0 con 0x7f6578114600 2024-09-21T03:08:00.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.155+0000 7f6585537740 1 Processor -- start 2024-09-21T03:08:00.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6575ffb640 1 -- 172.21.15.16:0/461693526 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f657002fd60 con 0x7f6578114600 2024-09-21T03:08:00.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 -- start start 2024-09-21T03:08:00.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73be6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73be6eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d73b5ce60 con 0x556d7383f350 2024-09-21T03:08:00.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556d73b4c290 con 0x556d7381e670 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.157+0000 7f6585537740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d73b500a0 con 0x556d73816490 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47460/0 (socket says 172.21.15.16:47460) 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 -- 172.21.15.16:0/2002948600 learned_addr learned my addr 172.21.15.16:0/2002948600 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f6577fff640 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73be6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f6577fff640 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73be6eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f6575ffb640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 0x7f65680b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 -- 172.21.15.16:0/2002948600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 msgr2=0x556d73be6eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73be6eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 -- 172.21.15.16:0/2002948600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 msgr2=0x556d73be6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73be6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 -- 172.21.15.16:0/2002948600 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d738aa930 con 0x556d73816490 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f6575ffb640 1 -- 172.21.15.16:0/461693526 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f6570108ab0 con 0x7f6578114600 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.158+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f656c00bc90 tx=0x7f656c00f730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e9429cb283746480 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.159+0000 7f657d560640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 0x7f65680b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.159+0000 7f6562ffd640 1 -- 172.21.15.16:0/2002948600 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c015070 con 0x556d73816490 2024-09-21T03:08:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.159+0000 7f6562ffd640 1 -- 172.21.15.16:0/2002948600 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c0035f0 con 0x556d73816490 2024-09-21T03:08:00.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.159+0000 7f6562ffd640 1 -- 172.21.15.16:0/2002948600 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c0116a0 con 0x556d73816490 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 -- 172.21.15.16:0/2002948600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 msgr2=0x556d73be6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f657d560640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 0x7f65680b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6570005e30 tx=0x7f6570003f40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f656c00bc90 tx=0x7f656c00f730 comp rx=0 tx=0).stop 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 -- 172.21.15.16:0/2002948600 shutdown_connections 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73be6eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73be6970 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 --2- 172.21.15.16:0/2002948600 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73be6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 -- 172.21.15.16:0/2002948600 >> 172.21.15.16:0/2002948600 conn(0x556d73833740 msgr2=0x556d73be4320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 -- 172.21.15.16:0/2002948600 shutdown_connections 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.160+0000 7f6585537740 1 -- 172.21.15.16:0/2002948600 wait complete. 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.161+0000 7f6585537740 1 -- start start 2024-09-21T03:08:00.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.161+0000 7f6585537740 1 Processor -- start 2024-09-21T03:08:00.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d73816490 0x556d73bf2030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7381e670 0x556d73bf2570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556d739ad890 con 0x556d7383f350 2024-09-21T03:08:00.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556d73ae6d40 con 0x556d73816490 2024-09-21T03:08:00.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.162+0000 7f6585537740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556d73bf4cb0 con 0x556d7381e670 2024-09-21T03:08:00.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6585537740 10 client.0 _ll_register_callbacks cb 0x556d73b4ac80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49790/0 (socket says 172.21.15.16:49790) 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 -- 172.21.15.16:0/950568253 learned_addr learned my addr 172.21.15.16:0/950568253 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d73816490 0x556d73bf2030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f657cd5f640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7381e670 0x556d73bf2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 -- 172.21.15.16:0/950568253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7381e670 msgr2=0x556d73bf2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7381e670 0x556d73bf2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 -- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d73816490 msgr2=0x556d73bf2030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d73816490 0x556d73bf2030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.163+0000 7f6577fff640 1 -- 172.21.15.16:0/950568253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d73bfe5f0 con 0x556d7383f350 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6577fff640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f656400d370 tx=0x7f6564004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3218bc3fa37b9f8a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6557fff640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400de40 con 0x556d7383f350 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f6564014110 con 0x556d7383f350 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f656c0035f0 con 0x556d7383f350 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6564028050 con 0x556d7383f350 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6585537740 10 client.16242 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:00.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.164+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556d73c00fb0 con 0x556d7383f350 2024-09-21T03:08:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f656c0035f0 con 0x556d7383f350 2024-09-21T03:08:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f6564003b50 con 0x556d7383f350 2024-09-21T03:08:00.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 10 client.16242 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 10 client.16242 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 20 client.16242 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d737fdcd0 0x556d73c04470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 4 client.16242 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.165+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556d73b5edb0 con 0x556d737fdcd0 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.166+0000 7f657cd5f640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d737fdcd0 0x556d73c04470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.166+0000 7f6557fff640 10 client.16242 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:00.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.166+0000 7f657cd5f640 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d737fdcd0 0x556d73c04470 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4e22fd287de57ac server_cookie=4ac360b816f97079 in_seq=0 out_seq=0 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.167+0000 7f6557fff640 1 -- 172.21.15.16:0/950568253 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556d73b5edb0 con 0x556d737fdcd0 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.167+0000 7f6557fff640 10 client.16242 handle_command_reply: tid=0 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.167+0000 7f6585537740 1 client.16242 shutdown 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 msgr2=0x556d73bfe030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f656400d370 tx=0x7f6564004730 comp rx=0 tx=0).stop 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 shutdown_connections 2024-09-21T03:08:00.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d737fdcd0 0x556d73c04470 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bfe030 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d7381e670 0x556d73bf2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 --2- 172.21.15.16:0/950568253 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d73816490 0x556d73bf2030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 >> 172.21.15.16:0/950568253 conn(0x556d73833740 msgr2=0x556d73be95c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.168+0000 7f6557fff640 0 client.16242 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:00.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.170+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 shutdown_connections 2024-09-21T03:08:00.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.170+0000 7f6585537740 1 -- 172.21.15.16:0/950568253 wait complete. 2024-09-21T03:08:00.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.170+0000 7f6585537740 20 client.16242 trim_cache size 0 max 16384 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.214+0000 7f6585537740 1 Processor -- start 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.214+0000 7f6585537740 1 -- start start 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73bfdf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73bec550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00c800 con 0x556d7383f350 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00f060 con 0x556d7381e670 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6585537740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00ce50 con 0x556d73816490 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.217 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6577fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49798/0 (socket says 172.21.15.16:49798) 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6577fff640 1 -- 172.21.15.16:0/1594195284 learned_addr learned my addr 172.21.15.16:0/1594195284 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.215+0000 7f6577fff640 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73bfdf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.216+0000 7f6577fff640 1 -- 172.21.15.16:0/1594195284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 msgr2=0x556d73bfdf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.216+0000 7f6577fff640 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73bfdf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.216+0000 7f6577fff640 1 -- 172.21.15.16:0/1594195284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 msgr2=0x556d73bec550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.216+0000 7f6577fff640 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73bec550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.216+0000 7f6577fff640 1 -- 172.21.15.16:0/1594195284 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556d73be8b60 con 0x556d7383f350 2024-09-21T03:08:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.217+0000 7f6577fff640 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6564001eb0 tx=0x7f65640046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254a6850dae40251 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.218+0000 7f65627fc640 1 -- 172.21.15.16:0/1594195284 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656400cd00 con 0x556d7383f350 2024-09-21T03:08:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.218+0000 7f65627fc640 1 -- 172.21.15.16:0/1594195284 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564004480 con 0x556d7383f350 2024-09-21T03:08:00.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.218+0000 7f65627fc640 1 -- 172.21.15.16:0/1594195284 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6564012e20 con 0x556d7383f350 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.219+0000 7f6585537740 1 -- 172.21.15.16:0/1594195284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 msgr2=0x556d73bf39c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.219+0000 7f6585537740 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f6564001eb0 tx=0x7f65640046c0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.220+0000 7f6585537740 1 -- 172.21.15.16:0/1594195284 shutdown_connections 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.220+0000 7f6585537740 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d7383f350 0x556d73bf39c0 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.220+0000 7f6585537740 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7381e670 0x556d73bec550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.220+0000 7f6585537740 1 --2- 172.21.15.16:0/1594195284 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d73816490 0x556d73bfdf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.220+0000 7f6585537740 1 -- 172.21.15.16:0/1594195284 >> 172.21.15.16:0/1594195284 conn(0x556d73833740 msgr2=0x556d73be7dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.228+0000 7f6585537740 1 -- 172.21.15.16:0/1594195284 shutdown_connections 2024-09-21T03:08:00.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.228+0000 7f6585537740 1 -- 172.21.15.16:0/1594195284 wait complete. 2024-09-21T03:08:00.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.229+0000 7f6585537740 1 -- start start 2024-09-21T03:08:00.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.229+0000 7f6585537740 1 Processor -- start 2024-09-21T03:08:00.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d73816490 0x556d73bf6110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d737d3080 0x556d73bf6b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00d520 con 0x556d73816490 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00e990 con 0x556d7383f350 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f655c00dbf0 con 0x556d737d3080 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.230+0000 7f6585537740 10 client.0 _ll_register_callbacks cb 0x7f656400cb70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.231+0000 7f657cd5f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.231+0000 7f657cd5f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53162/0 (socket says 172.21.15.16:53162) 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.231+0000 7f657cd5f640 1 -- 172.21.15.16:0/2022514129 learned_addr learned my addr 172.21.15.16:0/2022514129 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.231+0000 7f6577fff640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d73816490 0x556d73bf6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.231+0000 7f6577fff640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d737d3080 0x556d73bf6b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.232+0000 7f657cd5f640 1 -- 172.21.15.16:0/2022514129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d737d3080 msgr2=0x556d73bf6b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.232+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d737d3080 0x556d73bf6b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.232+0000 7f657cd5f640 1 -- 172.21.15.16:0/2022514129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d73816490 msgr2=0x556d73bf6110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.232+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d73816490 0x556d73bf6110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.232+0000 7f657cd5f640 1 -- 172.21.15.16:0/2022514129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556d73c7f080 con 0x556d7383f350 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.233+0000 7f657cd5f640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f656c00bc90 tx=0x7f656c00b2e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6221e85eb7b733b8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.235+0000 7f6557fff640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.235+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f656c015050 con 0x556d7383f350 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.235+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f656c005240 con 0x556d7383f350 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.235+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6564004480 con 0x556d7383f350 2024-09-21T03:08:00.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.235+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f656c033050 con 0x556d7383f350 2024-09-21T03:08:00.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.236+0000 7f6585537740 10 client.25904 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:00.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.236+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556d73c67080 con 0x556d7383f350 2024-09-21T03:08:00.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.236+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6564004480 con 0x556d7383f350 2024-09-21T03:08:00.241 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.237+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f656c025470 con 0x556d7383f350 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.238+0000 7f6585537740 10 client.25904 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.238+0000 7f6585537740 10 client.25904 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.238+0000 7f6585537740 20 client.25904 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.239+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d73c67450 0x556d73c69870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.239+0000 7f6585537740 4 client.25904 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.239+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556d73b5a860 con 0x556d73c67450 2024-09-21T03:08:00.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.239+0000 7f6577fff640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d73c67450 0x556d73c69870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.241+0000 7f6577fff640 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d73c67450 0x556d73c69870 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1899000e6741ca93 server_cookie=c20dc679248ea184 in_seq=0 out_seq=0 2024-09-21T03:08:00.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.241+0000 7f6557fff640 10 client.25904 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:00.250 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.248+0000 7f6557fff640 1 -- 172.21.15.16:0/2022514129 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556d73b5a860 con 0x556d73c67450 2024-09-21T03:08:00.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.249+0000 7f6557fff640 10 client.25904 handle_command_reply: tid=0 2024-09-21T03:08:00.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.249+0000 7f6585537740 1 client.25904 shutdown 2024-09-21T03:08:00.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.251+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 msgr2=0x556d73bf6650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.253 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.251+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f656c00bc90 tx=0x7f656c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.251+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 shutdown_connections 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.251+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556d73c67450 0x556d73c69870 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.252+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556d737d3080 0x556d73bf6b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.252+0000 7f6557fff640 0 client.25904 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.252+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556d7383f350 0x556d73bf6650 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.252+0000 7f6585537740 1 --2- 172.21.15.16:0/2022514129 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556d73816490 0x556d73bf6110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.254 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.252+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 >> 172.21.15.16:0/2022514129 conn(0x556d73833740 msgr2=0x556d73be7d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.254+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 shutdown_connections 2024-09-21T03:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.254+0000 7f6585537740 1 -- 172.21.15.16:0/2022514129 wait complete. 2024-09-21T03:08:00.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.255+0000 7f6585537740 20 client.25904 trim_cache size 0 max 16384 2024-09-21T03:08:00.258 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:00.258 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:08:00.258 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "2babef10-ec09-4cdf-987d-a41606a60c7f", 2024-09-21T03:08:00.258 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:08:00.258 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.257+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 msgr2=0x7f65680b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.257+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 0x7f65680b8c10 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f6570005e30 tx=0x7f6570003f40 comp rx=0 tx=0).stop 2024-09-21T03:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 msgr2=0x7f65781a62d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:00.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f6570005cc0 tx=0x7f657002f380 comp rx=0 tx=0).stop 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 shutdown_connections 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f65680b6700 0x7f65680b8c10 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6578114600 0x7f65781a62d0 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6578111ca0 0x7f65781a1030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 --2- 172.21.15.16:0/461693526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6578101ff0 0x7f65781a0a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 >> 172.21.15.16:0/461693526 conn(0x7f6578077cc0 msgr2=0x7f65780fece0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:00.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.258+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 shutdown_connections 2024-09-21T03:08:00.261 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:00.259+0000 7f657efc3640 1 -- 172.21.15.16:0/461693526 wait complete. 2024-09-21T03:08:00.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:00 smithi018 ceph-mon[34587]: pgmap v552: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 386 MiB/s wr, 778 op/s 2024-09-21T03:08:00.958 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:00 smithi016 ceph-mon[31253]: pgmap v552: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 386 MiB/s wr, 778 op/s 2024-09-21T03:08:00.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:00 smithi123 ceph-mon[34645]: pgmap v552: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 386 MiB/s wr, 778 op/s 2024-09-21T03:08:01.690 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:08:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:01 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:08:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:01 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:01 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:08:01.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:01 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:02.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:01 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:08:02.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:01 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:02.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:01 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:08:02.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:01 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:02.104 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:01 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:08:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:01 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:01 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:08:02.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:01 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:02.906 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:02 smithi016 ceph-mon[31253]: pgmap v553: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 410 MiB/s wr, 789 op/s 2024-09-21T03:08:02.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:02 smithi123 ceph-mon[34645]: pgmap v553: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 410 MiB/s wr, 789 op/s 2024-09-21T03:08:03.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 -- 172.21.15.16:0/952530688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 msgr2=0x7f6bb41087c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/952530688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb41087c0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4009900 tx=0x7f6ba402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 -- 172.21.15.16:0/952530688 shutdown_connections 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/952530688 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb41087c0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/952530688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4108d00 0x7f6bb410f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/952530688 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4102a70 0x7f6bb4102e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 -- 172.21.15.16:0/952530688 >> 172.21.15.16:0/952530688 conn(0x7f6bb4079730 msgr2=0x7f6bb4079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 -- 172.21.15.16:0/952530688 shutdown_connections 2024-09-21T03:08:03.021 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.019+0000 7f6bbb42c640 1 -- 172.21.15.16:0/952530688 wait complete. 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 Processor -- start 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 -- start start 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4102a70 0x7f6bb410ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4108d00 0x7f6bb4110a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113a90 con 0x7f6bb41083b0 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113910 con 0x7f6bb4108d00 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bbb42c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb4113c10 con 0x7f6bb4102a70 2024-09-21T03:08:03.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49830/0 (socket says 172.21.15.16:49830) 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 -- 172.21.15.16:0/1923471096 learned_addr learned my addr 172.21.15.16:0/1923471096 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4102a70 0x7f6bb410ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4108d00 0x7f6bb4110a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 -- 172.21.15.16:0/1923471096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4102a70 msgr2=0x7f6bb410ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4102a70 0x7f6bb410ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 -- 172.21.15.16:0/1923471096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4108d00 msgr2=0x7f6bb4110a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.020+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4108d00 0x7f6bb4110a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6bb89c7640 1 -- 172.21.15.16:0/1923471096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bb41a8800 con 0x7f6bb41083b0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4032fd0 tx=0x7f6ba4031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6baa7fc640 1 -- 172.21.15.16:0/1923471096 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba403f030 con 0x7f6bb41083b0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bb41a8a90 con 0x7f6bb41083b0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6baa7fc640 1 -- 172.21.15.16:0/1923471096 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba4004490 con 0x7f6bb41083b0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6baa7fc640 1 -- 172.21.15.16:0/1923471096 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4035070 con 0x7f6bb41083b0 2024-09-21T03:08:03.024 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.021+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bb41a8fc0 con 0x7f6bb41083b0 2024-09-21T03:08:03.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 Processor -- start 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 -- start start 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6baa7fc640 1 -- 172.21.15.16:0/1923471096 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6ba4042030 con 0x7f6bb41083b0 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 0x563a06c31b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c320d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a06ba7e50 con 0x563a06869390 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563a06b97280 con 0x563a068611b0 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a06b9b090 con 0x563a0688a070 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bb99c9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bb99c9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:47532/0 (socket says 172.21.15.16:47532) 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bb99c9640 1 -- 172.21.15.16:0/3851328111 learned_addr learned my addr 172.21.15.16:0/3851328111 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 0x563a06c31b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.024+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c320d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 -- 172.21.15.16:0/3851328111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 msgr2=0x563a06c31b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 0x563a06c31b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 -- 172.21.15.16:0/3851328111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 msgr2=0x563a06c320d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c320d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 -- 172.21.15.16:0/3851328111 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a068f5650 con 0x563a0688a070 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 0x563a06c31b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009f50 tx=0x7f6bac0118d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7277645f2ee15e1e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6baa7fc640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 0x7f6ba00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6b9f7fe640 1 -- 172.21.15.16:0/3851328111 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac004d20 con 0x563a0688a070 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6b9f7fe640 1 -- 172.21.15.16:0/3851328111 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bac004050 con 0x563a0688a070 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6b9f7fe640 1 -- 172.21.15.16:0/3851328111 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac005220 con 0x563a0688a070 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 0x7f6ba00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.028 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.025+0000 7f6baa7fc640 1 -- 172.21.15.16:0/1923471096 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f6ba41076c0 con 0x7f6bb41083b0 2024-09-21T03:08:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 -- 172.21.15.16:0/3851328111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 msgr2=0x563a06c31650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 0x7f6ba00b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x563a06c302d0 tx=0x7f6bb0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f6bac009f50 tx=0x7f6bac0118d0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 -- 172.21.15.16:0/3851328111 shutdown_connections 2024-09-21T03:08:03.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c320d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068611b0 0x563a06c31b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/3851328111 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c31650 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 -- 172.21.15.16:0/3851328111 >> 172.21.15.16:0/3851328111 conn(0x563a0687e460 msgr2=0x563a06c2f540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 -- 172.21.15.16:0/3851328111 shutdown_connections 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.026+0000 7f6bc19a0740 1 -- 172.21.15.16:0/3851328111 wait complete. 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 -- start start 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 Processor -- start 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 0x563a06c3ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c3d230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c49120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x563a069f8700 con 0x563a068611b0 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x563a06b32110 con 0x563a06869390 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x563a06c3fe30 con 0x563a0688a070 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bc19a0740 10 client.0 _ll_register_callbacks cb 0x563a06b4b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:03.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 0x563a06c3ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 0x563a06c3ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49854/0 (socket says 172.21.15.16:49854) 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bb91c8640 1 -- 172.21.15.16:0/1840101133 learned_addr learned my addr 172.21.15.16:0/1840101133 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.027+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c49120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c3d230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 -- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 msgr2=0x563a06c3d230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c3d230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 -- 172.21.15.16:0/1840101133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 msgr2=0x563a06c3ccf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 0x563a06c3ccf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 -- 172.21.15.16:0/1840101133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a06b5fc40 con 0x563a0688a070 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c49120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004750 tx=0x7f6bac004430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=761ab25cccfee746 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6b9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bac028020 con 0x563a0688a070 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f6bac005040 con 0x563a0688a070 2024-09-21T03:08:03.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bac004050 con 0x563a0688a070 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6bac009be0 con 0x563a0688a070 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bc19a0740 10 client.25900 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.028+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563a06c4bb20 con 0x563a0688a070 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bac004050 con 0x563a0688a070 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f6bac022630 con 0x563a0688a070 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 10 client.25900 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 10 client.25900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 20 client.25900 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a068489f0 0x563a06c4ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 4 client.25900 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a06ba9da0 con 0x563a068489f0 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.029+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a068489f0 0x563a06c4ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.030+0000 7f6b9cff9640 10 client.25900 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:03.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.030+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a068489f0 0x563a06c4ef90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4cb768280751e8da server_cookie=fd5070aee2edf58f in_seq=0 out_seq=0 2024-09-21T03:08:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6b9cff9640 1 -- 172.21.15.16:0/1840101133 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x563a06ba9da0 con 0x563a068489f0 2024-09-21T03:08:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6b9cff9640 10 client.25900 handle_command_reply: tid=0 2024-09-21T03:08:03.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 client.25900 shutdown 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 msgr2=0x563a06c49120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c49120 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f6bac004750 tx=0x7f6bac004430 comp rx=0 tx=0).stop 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 shutdown_connections 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a068489f0 0x563a06c4ef90 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a0688a070 0x563a06c49120 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c3d230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6b9cff9640 0 client.25900 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/1840101133 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068611b0 0x563a06c3ccf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 >> 172.21.15.16:0/1840101133 conn(0x563a0687e460 msgr2=0x563a06c347e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 shutdown_connections 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 1 -- 172.21.15.16:0/1840101133 wait complete. 2024-09-21T03:08:03.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.031+0000 7f6bc19a0740 20 client.25900 trim_cache size 0 max 16384 2024-09-21T03:08:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 Processor -- start 2024-09-21T03:08:03.055 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 -- start start 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c49090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c39ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41afa10 con 0x563a06869390 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b2270 con 0x563a068489f0 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.054+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b0060 con 0x563a068611b0 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:53186/0 (socket says 172.21.15.16:53186) 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb99c9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c39ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 -- 172.21.15.16:0/2539771869 learned_addr learned my addr 172.21.15.16:0/2539771869 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c49090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 -- 172.21.15.16:0/2539771869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 msgr2=0x563a06c49090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c49090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 -- 172.21.15.16:0/2539771869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 msgr2=0x563a06c39ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c39ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 -- 172.21.15.16:0/2539771869 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a06c3dfb0 con 0x563a068489f0 2024-09-21T03:08:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.055+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00099b0 tx=0x7f6bb001b000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77c55f4f1ed4bcef server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.056+0000 7f6b9effd640 1 -- 172.21.15.16:0/2539771869 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb0036070 con 0x563a068489f0 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.056+0000 7f6b9effd640 1 -- 172.21.15.16:0/2539771869 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb001dc20 con 0x563a068489f0 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.056+0000 7f6b9effd640 1 -- 172.21.15.16:0/2539771869 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb002a650 con 0x563a068489f0 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2539771869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 msgr2=0x563a06c33040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6bb00099b0 tx=0x7f6bb001b000 comp rx=0 tx=0).stop 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2539771869 shutdown_connections 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a06869390 0x563a06c39ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c49090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2539771869 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a068489f0 0x563a06c33040 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2539771869 >> 172.21.15.16:0/2539771869 conn(0x563a0687e460 msgr2=0x563a06c3d110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2539771869 shutdown_connections 2024-09-21T03:08:03.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.057+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2539771869 wait complete. 2024-09-21T03:08:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 -- start start 2024-09-21T03:08:03.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 Processor -- start 2024-09-21T03:08:03.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c50550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c50a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b0730 con 0x563a068489f0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b1ba0 con 0x563a06869390 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6bb41b0e00 con 0x563a068611b0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bc19a0740 10 client.0 _ll_register_callbacks cb 0x7f6bb0009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bb89c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c50550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bb91c8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:49866/0 (socket says 172.21.15.16:49866) 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bb99c9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c50a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.058+0000 7f6bb91c8640 1 -- 172.21.15.16:0/2813982814 learned_addr learned my addr 172.21.15.16:0/2813982814 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 -- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 msgr2=0x563a06c50a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c50a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 -- 172.21.15.16:0/2813982814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 msgr2=0x563a06c50010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 -- 172.21.15.16:0/2813982814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a06cb8710 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb99c9640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c50a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bb89c7640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c50550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4104e00 tx=0x7f6ba4003f60 comp rx=0 tx=0).ready entity=mon.1 client_cookie=993c74403af904df server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6b9cff9640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ba4049050 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f6ba403bc30 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6bb001dc20 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6ba40cd0c0 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.059+0000 7f6bc19a0740 10 client.25906 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x563a06cb9a00 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ba40d1fe0 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f6ba404c060 con 0x563a068611b0 2024-09-21T03:08:03.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 10 client.25906 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 10 client.25906 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 20 client.25906 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a0688a070 0x563a06cbc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 4 client.25906 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7f6bb001dc20 con 0x563a0688a070 2024-09-21T03:08:03.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.060+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a0688a070 0x563a06cbc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.061+0000 7f6b9cff9640 10 client.25906 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.061+0000 7f6bb91c8640 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a0688a070 0x563a06cbc170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6580757a93415545 server_cookie=45582299a52fda99 in_seq=0 out_seq=0 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.061+0000 7f6b9cff9640 1 -- 172.21.15.16:0/2813982814 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4066 (crc 0 0 0) 0x7f6bb001dc20 con 0x563a0688a070 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.061+0000 7f6b9cff9640 10 client.25906 handle_command_reply: tid=0 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.061+0000 7f6bc19a0740 1 client.25906 shutdown 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 msgr2=0x563a06c50550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c50550 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4104e00 tx=0x7f6ba4003f60 comp rx=0 tx=0).stop 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 shutdown_connections 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x563a0688a070 0x563a06cbc170 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6b9cff9640 0 client.25906 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x563a06869390 0x563a06c50a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563a068611b0 0x563a06c50550 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 --2- 172.21.15.16:0/2813982814 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x563a068489f0 0x563a06c50010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 >> 172.21.15.16:0/2813982814 conn(0x563a0687e460 msgr2=0x563a06c32540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 shutdown_connections 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 1 -- 172.21.15.16:0/2813982814 wait complete. 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.062+0000 7f6bc19a0740 20 client.25906 trim_cache size 0 max 16384 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:03.066 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.067 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:08:03.068 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.069 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.070 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.071 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:08:03.072 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.073 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:03.074 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:03.075 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 msgr2=0x7f6ba00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 0x7f6ba00b8c10 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x563a06c302d0 tx=0x7f6bb0005f90 comp rx=0 tx=0).stop 2024-09-21T03:08:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 msgr2=0x7f6bb4110500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f6ba4032fd0 tx=0x7f6ba4031a20 comp rx=0 tx=0).stop 2024-09-21T03:08:03.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 shutdown_connections 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6ba00b6700 0x7f6ba00b8c10 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6bb41083b0 0x7f6bb4110500 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6bb4108d00 0x7f6bb4110a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 --2- 172.21.15.16:0/1923471096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bb4102a70 0x7f6bb410ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 >> 172.21.15.16:0/1923471096 conn(0x7f6bb4079730 msgr2=0x7f6bb4105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 shutdown_connections 2024-09-21T03:08:03.077 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:03.063+0000 7f6bbb42c640 1 -- 172.21.15.16:0/1923471096 wait complete. 2024-09-21T03:08:03.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:02 smithi018 ceph-mon[34587]: pgmap v553: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 410 MiB/s wr, 789 op/s 2024-09-21T03:08:03.961 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:03 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:08:04.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:03 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:08:04.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:03 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:08:04.567 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:08:04.568 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:08:04.960 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:05.090 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:04 smithi016 ceph-mon[31253]: pgmap v554: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 324 MiB/s wr, 539 op/s 2024-09-21T03:08:05.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:04 smithi018 ceph-mon[34587]: pgmap v554: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 324 MiB/s wr, 539 op/s 2024-09-21T03:08:05.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:04 smithi123 ceph-mon[34645]: pgmap v554: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.4 MiB/s rd, 324 MiB/s wr, 539 op/s 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 -- 172.21.15.16:0/310496244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 msgr2=0x7f104c1061a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 --2- 172.21.15.16:0/310496244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c1061a0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f1040009900 tx=0x7f104002a440 comp rx=0 tx=0).stop 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 -- 172.21.15.16:0/310496244 shutdown_connections 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 --2- 172.21.15.16:0/310496244 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c10c020 0x7f104c112850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 --2- 172.21.15.16:0/310496244 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c10bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 --2- 172.21.15.16:0/310496244 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c1061a0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.653+0000 7f1052f58640 1 -- 172.21.15.16:0/310496244 >> 172.21.15.16:0/310496244 conn(0x7f104c1017e0 msgr2=0x7f104c103c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:05.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.654+0000 7f1052f58640 1 -- 172.21.15.16:0/310496244 shutdown_connections 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.654+0000 7f1052f58640 1 -- 172.21.15.16:0/310496244 wait complete. 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.655+0000 7f1052f58640 1 Processor -- start 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.655+0000 7f1052f58640 1 -- start start 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.655+0000 7f1052f58640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c10fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.655+0000 7f1052f58640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f1052f58640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c10c020 0x7f104c1107a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f1052f58640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f104c115e10 con 0x7f104c105d90 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f1052f58640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f104c115c90 con 0x7f104c10b6d0 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f1052f58640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f104c115f90 con 0x7f104c10c020 2024-09-21T03:08:05.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f104bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f104bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59832/0 (socket says 172.21.15.16:59832) 2024-09-21T03:08:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.656+0000 7f104bfff640 1 -- 172.21.15.16:0/3575771736 learned_addr learned my addr 172.21.15.16:0/3575771736 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:05.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f1050cf4640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c10fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f104bfff640 1 -- 172.21.15.16:0/3575771736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c10c020 msgr2=0x7f104c1107a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f104bfff640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c10c020 0x7f104c1107a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f104bfff640 1 -- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 msgr2=0x7f104c10fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f104bfff640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c10fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.657+0000 7f104bfff640 1 -- 172.21.15.16:0/3575771736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f104c1a24d0 con 0x7f104c10b6d0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.658+0000 7f104bfff640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f103800b450 tx=0x7f103800b910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.658+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1038015020 con 0x7f104c10b6d0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.658+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1038011d40 con 0x7f104c10b6d0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.658+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f103800f340 con 0x7f104c10b6d0 2024-09-21T03:08:05.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.658+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f104c1a27c0 con 0x7f104c10b6d0 2024-09-21T03:08:05.661 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.659+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f104c1a2cd0 con 0x7f104c10b6d0 2024-09-21T03:08:05.663 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.661+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f104c106220 con 0x7f104c10b6d0 2024-09-21T03:08:05.664 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.663+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1038018020 con 0x7f104c10b6d0 2024-09-21T03:08:05.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.664+0000 7f1049ffb640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 0x7f103c0b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.665+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f10380e7180 con 0x7f104c10b6d0 2024-09-21T03:08:05.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.665+0000 7f1050cf4640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 0x7f103c0b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:05.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.666+0000 7f1050cf4640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 0x7f103c0b8c90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1040006010 tx=0x7f1040031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:05.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:05.669+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f103801f020 con 0x7f104c10b6d0 2024-09-21T03:08:06.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.035+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f104c1a3900 con 0x7f104c10b6d0 2024-09-21T03:08:06.037 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.035+0000 7f1049ffb640 1 -- 172.21.15.16:0/3575771736 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f103800d040 con 0x7f104c10b6d0 2024-09-21T03:08:06.037 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:08:06.037 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:08:06.038 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:08:06.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 msgr2=0x7f103c0b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:06.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 0x7f103c0b8c90 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f1040006010 tx=0x7f1040031400 comp rx=0 tx=0).stop 2024-09-21T03:08:06.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 msgr2=0x7f104c110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:06.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f103800b450 tx=0x7f103800b910 comp rx=0 tx=0).stop 2024-09-21T03:08:06.039 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 shutdown_connections 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f103c0b6780 0x7f103c0b8c90 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f104c10c020 0x7f104c1107a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f104c10b6d0 0x7f104c110260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 --2- 172.21.15.16:0/3575771736 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f104c105d90 0x7f104c10fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 >> 172.21.15.16:0/3575771736 conn(0x7f104c1017e0 msgr2=0x7f104c103c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 shutdown_connections 2024-09-21T03:08:06.040 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:06.038+0000 7f1052f58640 1 -- 172.21.15.16:0/3575771736 wait complete. 2024-09-21T03:08:06.898 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:08:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:06 smithi018 ceph-mon[34587]: pgmap v555: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 416 MiB/s wr, 700 op/s 2024-09-21T03:08:07.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:06 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3575771736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:06 smithi123 ceph-mon[34645]: pgmap v555: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 416 MiB/s wr, 700 op/s 2024-09-21T03:08:07.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:06 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3575771736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:07.263 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:07.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:06 smithi016 ceph-mon[31253]: pgmap v555: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 416 MiB/s wr, 700 op/s 2024-09-21T03:08:07.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:06 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3575771736' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 -- 172.21.15.16:0/3402838388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c81053f0 msgr2=0x7f68c8105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 --2- 172.21.15.16:0/3402838388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c81053f0 0x7f68c8105800 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f68c4009900 tx=0x7f68c402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 -- 172.21.15.16:0/3402838388 shutdown_connections 2024-09-21T03:08:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 --2- 172.21.15.16:0/3402838388 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81020f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 --2- 172.21.15.16:0/3402838388 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c8102630 0x7f68c8106190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 --2- 172.21.15.16:0/3402838388 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c81053f0 0x7f68c8105800 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.855+0000 7f68d02c7640 1 -- 172.21.15.16:0/3402838388 >> 172.21.15.16:0/3402838388 conn(0x7f68c8077c60 msgr2=0x7f68c80780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.857+0000 7f68d02c7640 1 -- 172.21.15.16:0/3402838388 shutdown_connections 2024-09-21T03:08:07.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.857+0000 7f68d02c7640 1 -- 172.21.15.16:0/3402838388 wait complete. 2024-09-21T03:08:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.858+0000 7f68d02c7640 1 Processor -- start 2024-09-21T03:08:07.860 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.858+0000 7f68d02c7640 1 -- start start 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c8102630 0x7f68c81a1130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 0x7f68c81a63d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68c811a080 con 0x7f68c8101ce0 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f68c8119f00 con 0x7f68c81053f0 2024-09-21T03:08:07.861 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68d02c7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c811a200 con 0x7f68c8102630 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68ce063640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68ce063640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58466/0 (socket says 172.21.15.16:58466) 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68ce063640 1 -- 172.21.15.16:0/543593037 learned_addr learned my addr 172.21.15.16:0/543593037 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.859+0000 7f68ce864640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 0x7f68c81a63d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68cd862640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c8102630 0x7f68c81a1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.862 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 -- 172.21.15.16:0/543593037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c8102630 msgr2=0x7f68c81a1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c8102630 0x7f68c81a1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 -- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 msgr2=0x7f68c81a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 -- 172.21.15.16:0/543593037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68c81a6b00 con 0x7f68c81053f0 2024-09-21T03:08:07.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce063640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.860+0000 7f68ce864640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 0x7f68c81a63d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68c000da10 tx=0x7f68c000ded0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.861+0000 7f68bf7fe640 1 -- 172.21.15.16:0/543593037 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c000bd70 con 0x7f68c81053f0 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.861+0000 7f68bf7fe640 1 -- 172.21.15.16:0/543593037 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c0003e20 con 0x7f68c81053f0 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.861+0000 7f68bf7fe640 1 -- 172.21.15.16:0/543593037 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00106c0 con 0x7f68c81053f0 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.861+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68c81a6df0 con 0x7f68c81053f0 2024-09-21T03:08:07.864 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.861+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68c81ad550 con 0x7f68c81053f0 2024-09-21T03:08:07.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.865+0000 7f68d683b740 1 Processor -- start 2024-09-21T03:08:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.866+0000 7f68bf7fe640 1 -- 172.21.15.16:0/543593037 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f68c0004270 con 0x7f68c81053f0 2024-09-21T03:08:07.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.866+0000 7f68d683b740 1 -- start start 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.866+0000 7f68d683b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ec86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.866+0000 7f68d683b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68d683b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b00390 0x561d14ec9120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68d683b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d14e3eea0 con 0x561d14b21070 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68d683b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d14e2e2d0 con 0x561d14af81b0 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68d683b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561d14e320e0 con 0x561d14b00390 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59850/0 (socket says 172.21.15.16:59850) 2024-09-21T03:08:07.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 -- 172.21.15.16:0/1667356913 learned_addr learned my addr 172.21.15.16:0/1667356913 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68cd862640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b00390 0x561d14ec9120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68cd862640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ec86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 -- 172.21.15.16:0/1667356913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b00390 msgr2=0x561d14ec9120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b00390 0x561d14ec9120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 -- 172.21.15.16:0/1667356913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 msgr2=0x561d14ec86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ec86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68ce063640 1 -- 172.21.15.16:0/1667356913 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d14b8c650 con 0x561d14af81b0 2024-09-21T03:08:07.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.867+0000 7f68cd862640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ec86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68bf7fe640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 0x7f68b40b8c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68ce063640 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000c00 tx=0x7f68c40314a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7f33478a0270169c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68bcd9a640 1 -- 172.21.15.16:0/1667356913 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4035040 con 0x561d14af81b0 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68cd862640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 0x7f68b40b8c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68bcd9a640 1 -- 172.21.15.16:0/1667356913 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c4031890 con 0x561d14af81b0 2024-09-21T03:08:07.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.868+0000 7f68bf7fe640 1 -- 172.21.15.16:0/543593037 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f68c00e7ae0 con 0x7f68c81053f0 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.869+0000 7f68bcd9a640 1 -- 172.21.15.16:0/1667356913 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c40049e0 con 0x561d14af81b0 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 -- 172.21.15.16:0/1667356913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 msgr2=0x561d14ec8be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000c00 tx=0x7f68c40314a0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68cd862640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 0x7f68b40b8c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68c8105b70 tx=0x7f68b8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 -- 172.21.15.16:0/1667356913 shutdown_connections 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b00390 0x561d14ec9120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec8be0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 --2- 172.21.15.16:0/1667356913 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ec86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 -- 172.21.15.16:0/1667356913 >> 172.21.15.16:0/1667356913 conn(0x561d14b15460 msgr2=0x561d14ec6590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 -- 172.21.15.16:0/1667356913 shutdown_connections 2024-09-21T03:08:07.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.870+0000 7f68d683b740 1 -- 172.21.15.16:0/1667356913 wait complete. 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.871+0000 7f68d683b740 1 -- start start 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.871+0000 7f68d683b740 1 Processor -- start 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14b00390 0x561d14ed4220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ed4760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68ce063640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68ce063640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45542/0 (socket says 172.21.15.16:45542) 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68ce063640 1 -- 172.21.15.16:0/544511899 learned_addr learned my addr 172.21.15.16:0/544511899 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561d14c8f700 con 0x561d14b21070 2024-09-21T03:08:07.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561d14dc9160 con 0x561d14b00390 2024-09-21T03:08:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.872+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561d14ed6ec0 con 0x561d14af81b0 2024-09-21T03:08:07.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68d683b740 10 client.0 _ll_register_callbacks cb 0x561d14de29e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 -- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14b00390 msgr2=0x561d14ed4220 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14b00390 0x561d14ed4220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 -- 172.21.15.16:0/544511899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 msgr2=0x561d14ed4760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ed4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d14df6c90 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68ce063640 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68c4004550 tx=0x7f68c4004580 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3222923b2ed32cc7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68b1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.873+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c4031890 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68b1ffb640 1 -- 172.21.15.16:0/544511899 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c404e020 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68b1ffb640 1 -- 172.21.15.16:0/544511899 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f68c4003800 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68b1ffb640 1 -- 172.21.15.16:0/544511899 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f68c4059050 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68d683b740 10 client.25918 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=20}) -- 0x561d14ee28c0 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.874+0000 7f68b1ffb640 1 -- 172.21.15.16:0/544511899 <== mon.1 v2:172.21.15.123:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f68c4003a00 con 0x561d14af81b0 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 10 client.25918 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 10 client.25918 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 20 client.25918 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14adf9f0 0x561d14ee5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 4 client.25918 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561d14e40df0 con 0x561d14adf9f0 2024-09-21T03:08:07.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.875+0000 7f68cd862640 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14adf9f0 0x561d14ee5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.876+0000 7f68b1ffb640 10 client.25918 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.876+0000 7f68cd862640 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14adf9f0 0x561d14ee5ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696a520460b5b72d server_cookie=c083eb63a7994b39 in_seq=0 out_seq=0 2024-09-21T03:08:07.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68b1ffb640 1 -- 172.21.15.16:0/544511899 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561d14e40df0 con 0x561d14adf9f0 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68b1ffb640 10 client.25918 handle_command_reply: tid=0 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 client.25918 shutdown 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 msgr2=0x561d14ed3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68c4004550 tx=0x7f68c4004580 comp rx=0 tx=0).stop 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 shutdown_connections 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14adf9f0 0x561d14ee5ee0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b21070 0x561d14ed4760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14b00390 0x561d14ed4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 --2- 172.21.15.16:0/544511899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14af81b0 0x561d14ed3ce0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 >> 172.21.15.16:0/544511899 conn(0x561d14b15460 msgr2=0x561d14ec96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.877+0000 7f68b1ffb640 0 client.25918 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.878+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 shutdown_connections 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.878+0000 7f68d683b740 1 -- 172.21.15.16:0/544511899 wait complete. 2024-09-21T03:08:07.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.878+0000 7f68d683b740 20 client.25918 trim_cache size 0 max 16384 2024-09-21T03:08:07.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.905+0000 7f68d683b740 1 Processor -- start 2024-09-21T03:08:07.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.905+0000 7f68d683b740 1 -- start start 2024-09-21T03:08:07.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14ec5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14ec6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59870/0 (socket says 172.21.15.16:59870) 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68ce063640 1 -- 172.21.15.16:0/2801284801 learned_addr learned my addr 172.21.15.16:0/2801284801 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b3e00 con 0x561d14b00390 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68cd862640 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14ec5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b6660 con 0x561d14af81b0 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68ce864640 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14ec6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.909 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.906+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b4450 con 0x561d14b21070 2024-09-21T03:08:07.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 -- 172.21.15.16:0/2801284801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 msgr2=0x561d14ec9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.910 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 -- 172.21.15.16:0/2801284801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 msgr2=0x561d14ec5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14ec5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 -- 172.21.15.16:0/2801284801 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561d14eca990 con 0x561d14b21070 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68ce864640 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14ec6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e51c0 tx=0x7f68c00ad690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=83885f14ce154be3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68b3fff640 1 -- 172.21.15.16:0/2801284801 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c00ad3d0 con 0x561d14b21070 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68b3fff640 1 -- 172.21.15.16:0/2801284801 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68c00adfc0 con 0x561d14b21070 2024-09-21T03:08:07.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.907+0000 7f68b3fff640 1 -- 172.21.15.16:0/2801284801 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c0004da0 con 0x561d14b21070 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 msgr2=0x561d14ec6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14ec6100 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f68c00e51c0 tx=0x7f68c00ad690 comp rx=0 tx=0).stop 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 shutdown_connections 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14ec5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14ec6100 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 --2- 172.21.15.16:0/2801284801 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14ec9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 >> 172.21.15.16:0/2801284801 conn(0x561d14b15460 msgr2=0x561d14ec96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.909+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 shutdown_connections 2024-09-21T03:08:07.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.910+0000 7f68d683b740 1 -- 172.21.15.16:0/2801284801 wait complete. 2024-09-21T03:08:07.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.910+0000 7f68d683b740 1 -- start start 2024-09-21T03:08:07.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.910+0000 7f68d683b740 1 Processor -- start 2024-09-21T03:08:07.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.911+0000 7f68d683b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.911+0000 7f68d683b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14f49030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.911+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.911+0000 7f68ce063640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59876/0 (socket says 172.21.15.16:59876) 2024-09-21T03:08:07.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.911+0000 7f68ce063640 1 -- 172.21.15.16:0/3526081506 learned_addr learned my addr 172.21.15.16:0/3526081506 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14edbf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b4b20 con 0x561d14b00390 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b5f90 con 0x561d14af81b0 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68cd862640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14f49030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68ce864640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14edbf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.912+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f68c81b51f0 con 0x561d14b21070 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 -- 172.21.15.16:0/3526081506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 msgr2=0x561d14edbf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14edbf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 -- 172.21.15.16:0/3526081506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 msgr2=0x561d14f49030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14f49030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561d14edc600 con 0x561d14af81b0 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68d683b740 10 client.0 _ll_register_callbacks cb 0x7f68c4040070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:07.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68c00adfc0 con 0x561d14af81b0 2024-09-21T03:08:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68ce063640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000c00 tx=0x7f68c4005fd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5ffb4df7d78bdabd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.913+0000 7f68b1ffb640 10 client.25940 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:08:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.914+0000 7f68b1ffb640 1 -- 172.21.15.16:0/3526081506 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68c4062020 con 0x561d14af81b0 2024-09-21T03:08:07.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.914+0000 7f68b1ffb640 1 -- 172.21.15.16:0/3526081506 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(81..81 src has 1..81) ==== 8536+0+0 (secure 0 0 0) 0x7f68c405e690 con 0x561d14af81b0 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.914+0000 7f68b1ffb640 1 -- 172.21.15.16:0/3526081506 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f68c404e050 con 0x561d14af81b0 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.914+0000 7f68d683b740 10 client.25940 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.914+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({fsmap=20}) -- 0x561d14f500b0 con 0x561d14af81b0 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68b1ffb640 1 -- 172.21.15.16:0/3526081506 <== mon.2 v2:172.21.15.18:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f68c405e890 con 0x561d14af81b0 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 10 client.25940 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 10 client.25940 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 20 client.25940 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:07.918 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14ab4da0 0x561d14f52700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 4 client.25940 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:08:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561d14e3c8a0 con 0x561d14ab4da0 2024-09-21T03:08:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.915+0000 7f68cd862640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14ab4da0 0x561d14f52700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68cd862640 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14ab4da0 0x561d14f52700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5a2376b333ff8d72 server_cookie=1b2772b5891a6678 in_seq=0 out_seq=0 2024-09-21T03:08:07.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68b1ffb640 10 client.25940 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68b1ffb640 1 -- 172.21.15.16:0/3526081506 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561d14e3c8a0 con 0x561d14ab4da0 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68b1ffb640 10 client.25940 handle_command_reply: tid=0 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68d683b740 1 client.25940 shutdown 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 msgr2=0x561d14f48a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f68c4000c00 tx=0x7f68c4005fd0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 shutdown_connections 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.916+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561d14ab4da0 0x561d14f52700 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561d14b00390 0x561d14f49030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561d14b21070 0x561d14edbf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 --2- 172.21.15.16:0/3526081506 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561d14af81b0 0x561d14f48a90 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 >> 172.21.15.16:0/3526081506 conn(0x561d14b15460 msgr2=0x561d14ec9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68b1ffb640 0 client.25940 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 shutdown_connections 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 1 -- 172.21.15.16:0/3526081506 wait complete. 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.917+0000 7f68d683b740 20 client.25940 trim_cache size 0 max 16384 2024-09-21T03:08:07.921 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:08:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 msgr2=0x7f68b40b8c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 0x7f68b40b8c90 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f68c8105b70 tx=0x7f68b8005f90 comp rx=0 tx=0).stop 2024-09-21T03:08:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 msgr2=0x7f68c81a63d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 0x7f68c81a63d0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f68c000da10 tx=0x7f68c000ded0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.922 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 shutdown_connections 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f68b40b6780 0x7f68b40b8c90 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f68c81053f0 0x7f68c81a63d0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c8102630 0x7f68c81a1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 --2- 172.21.15.16:0/543593037 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f68c8101ce0 0x7f68c81a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.918+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 >> 172.21.15.16:0/543593037 conn(0x7f68c8077c60 msgr2=0x7f68c8100cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.919+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 shutdown_connections 2024-09-21T03:08:07.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:07.919+0000 7f68d02c7640 1 -- 172.21.15.16:0/543593037 wait complete. 2024-09-21T03:08:09.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:08 smithi018 ceph-mon[34587]: pgmap v556: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 MiB/s wr, 756 op/s 2024-09-21T03:08:09.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:08 smithi016 ceph-mon[31253]: pgmap v556: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 MiB/s wr, 756 op/s 2024-09-21T03:08:09.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:08 smithi123 ceph-mon[34645]: pgmap v556: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 415 MiB/s wr, 756 op/s 2024-09-21T03:08:09.913 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:08:09.914 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:08:10.083 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:09 smithi018 ceph-mon[34587]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:08:10.182 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:10.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:09 smithi123 ceph-mon[34645]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:08:10.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:09 smithi016 ceph-mon[31253]: osdmap e82: 12 total, 12 up, 12 in 2024-09-21T03:08:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 -- 172.21.15.16:0/3484321418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff728103b70 msgr2=0x7ff728103f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 --2- 172.21.15.16:0/3484321418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff728103b70 0x7ff728103f80 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7ff724009900 tx=0x7ff72402a410 comp rx=0 tx=0).stop 2024-09-21T03:08:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 -- 172.21.15.16:0/3484321418 shutdown_connections 2024-09-21T03:08:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 --2- 172.21.15.16:0/3484321418 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728109e00 0x7ff728110630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.700 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 --2- 172.21.15.16:0/3484321418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7281094b0 0x7ff7281098c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 --2- 172.21.15.16:0/3484321418 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff728103b70 0x7ff728103f80 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.697+0000 7ff73074a640 1 -- 172.21.15.16:0/3484321418 >> 172.21.15.16:0/3484321418 conn(0x7ff7280ff5e0 msgr2=0x7ff728101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.698+0000 7ff73074a640 1 -- 172.21.15.16:0/3484321418 shutdown_connections 2024-09-21T03:08:10.701 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.698+0000 7ff73074a640 1 -- 172.21.15.16:0/3484321418 wait complete. 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.698+0000 7ff73074a640 1 Processor -- start 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.698+0000 7ff73074a640 1 -- start start 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7281094b0 0x7ff72819aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 0x7ff72819fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff728113b10 con 0x7ff7281094b0 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff728113990 con 0x7ff728103b70 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff73074a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff728113c90 con 0x7ff728109e00 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72dce5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7281094b0 0x7ff72819aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59886/0 (socket says 172.21.15.16:59886) 2024-09-21T03:08:10.703 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 -- 172.21.15.16:0/2411391841 learned_addr learned my addr 172.21.15.16:0/2411391841 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72ece7640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 0x7ff72819fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 -- 172.21.15.16:0/2411391841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 msgr2=0x7ff72819fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 0x7ff72819fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 -- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7281094b0 msgr2=0x7ff72819aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7281094b0 0x7ff72819aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72e4e6640 1 -- 172.21.15.16:0/2411391841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7281a0410 con 0x7ff728103b70 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.699+0000 7ff72ece7640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 0x7ff72819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff724005ab0 tx=0x7ff724003c10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff71f7fe640 1 -- 172.21.15.16:0/2411391841 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72403f070 con 0x7ff728103b70 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff71f7fe640 1 -- 172.21.15.16:0/2411391841 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff724031b60 con 0x7ff728103b70 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7281a06a0 con 0x7ff728103b70 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff71f7fe640 1 -- 172.21.15.16:0/2411391841 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff724035030 con 0x7ff728103b70 2024-09-21T03:08:10.704 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.700+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7280755f0 con 0x7ff728103b70 2024-09-21T03:08:10.705 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.703+0000 7ff736cbe740 1 Processor -- start 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.703+0000 7ff736cbe740 1 -- start start 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.703+0000 7ff71f7fe640 1 -- 172.21.15.16:0/2411391841 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff724042070 con 0x7ff728103b70 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.703+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838b4eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff736cbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56248382ae60 con 0x56248350d350 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff736cbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56248381a290 con 0x5624834e4490 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff736cbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x56248381e0a0 con 0x5624834ec670 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72e4e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58548/0 (socket says 172.21.15.16:58548) 2024-09-21T03:08:10.707 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72e4e6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:59902/0 (socket says 172.21.15.16:59902) 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 -- 172.21.15.16:0/1523580029 learned_addr learned my addr 172.21.15.16:0/1523580029 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72dce5640 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838b4eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 -- 172.21.15.16:0/1523580029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 msgr2=0x5624838b4eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838b4eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 -- 172.21.15.16:0/1523580029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 msgr2=0x5624838b4970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72ece7640 1 -- 172.21.15.16:0/1523580029 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562483578930 con 0x56248350d350 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.704+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello state changed while learned_addr, mark_down or replacing must be happened just now 2024-09-21T03:08:10.708 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff72ece7640 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff7200117e0 tx=0x7ff720011ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e579877e057a61f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff71f7fe640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 0x7ff7140b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff71cff9640 1 -- 172.21.15.16:0/1523580029 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720011310 con 0x56248350d350 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff71cff9640 1 -- 172.21.15.16:0/1523580029 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff720003e20 con 0x56248350d350 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff71cff9640 1 -- 172.21.15.16:0/1523580029 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7200044f0 con 0x56248350d350 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 0x7ff7140b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.705+0000 7ff71f7fe640 1 -- 172.21.15.16:0/2411391841 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7ff724107b00 con 0x7ff728103b70 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 -- 172.21.15.16:0/1523580029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 msgr2=0x5624838b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff72e4e6640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 0x7ff7140b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff7241052c0 tx=0x7ff724031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.710 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7ff7200117e0 tx=0x7ff720011ca0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 -- 172.21.15.16:0/1523580029 shutdown_connections 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838b4eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838b4970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 --2- 172.21.15.16:0/1523580029 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b4430 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 -- 172.21.15.16:0/1523580029 >> 172.21.15.16:0/1523580029 conn(0x562483501740 msgr2=0x5624838b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 -- 172.21.15.16:0/1523580029 shutdown_connections 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.706+0000 7ff736cbe740 1 -- 172.21.15.16:0/1523580029 wait complete. 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 -- start start 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 Processor -- start 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 0x5624838c0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838c0550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x5624838cbf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56248367b890 con 0x5624834e4490 2024-09-21T03:08:10.711 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5624837b4d40 con 0x56248350d350 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5624838c2c10 con 0x5624834ec670 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff736cbe740 10 client.0 _ll_register_callbacks cb 0x562483818c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 0x5624838c0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 0x5624838c0010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58562/0 (socket says 172.21.15.16:58562) 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff72ece7640 1 -- 172.21.15.16:0/2033361535 learned_addr learned my addr 172.21.15.16:0/2033361535 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff72dce5640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838c0550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.707+0000 7ff72ece7640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x5624838cbf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 -- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 msgr2=0x5624838cbf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x5624838cbf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 -- 172.21.15.16:0/2033361535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 msgr2=0x5624838c0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 0x5624838c0010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.712 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 -- 172.21.15.16:0/2033361535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624838cc490 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff72dce5640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838c0550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff72806bab0 tx=0x7ff718009bf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a296eedd3bded6d9 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff711ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff718004460 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff720003e20 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.708+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7ff71800b770 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff718029050 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 10 client.25936 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5624838cede0 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff720003e20 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7ff718003b30 con 0x5624834ec670 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 10 client.25936 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 10 client.25936 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:10.715 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 20 client.25936 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834cbcd0 0x5624838d2280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 4 client.25936 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56248382cdb0 con 0x5624834cbcd0 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.709+0000 7ff72ece7640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834cbcd0 0x5624838d2280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.710+0000 7ff711ffb640 10 client.25936 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.710+0000 7ff72ece7640 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834cbcd0 0x5624838d2280 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d16c9262984e1f47 server_cookie=1a21115ef972feea in_seq=0 out_seq=0 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.710+0000 7ff711ffb640 1 -- 172.21.15.16:0/2033361535 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x56248382cdb0 con 0x5624834cbcd0 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.710+0000 7ff711ffb640 10 client.25936 handle_command_reply: tid=0 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.710+0000 7ff736cbe740 1 client.25936 shutdown 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 msgr2=0x5624838c0550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838c0550 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7ff72806bab0 tx=0x7ff718009bf0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 shutdown_connections 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834cbcd0 0x5624838d2280 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x5624838cbf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838c0550 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 --2- 172.21.15.16:0/2033361535 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834e4490 0x5624838c0010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff711ffb640 0 client.25936 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 >> 172.21.15.16:0/2033361535 conn(0x562483501740 msgr2=0x5624838b7610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 shutdown_connections 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 1 -- 172.21.15.16:0/2033361535 wait complete. 2024-09-21T03:08:10.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.711+0000 7ff736cbe740 20 client.25936 trim_cache size 0 max 16384 2024-09-21T03:08:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 Processor -- start 2024-09-21T03:08:10.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 -- start start 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838bb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281ab6b0 con 0x56248350d350 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281adf10 con 0x5624834e4490 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff736cbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281abd00 con 0x5624834ec670 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58574/0 (socket says 172.21.15.16:58574) 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 -- 172.21.15.16:0/508114197 learned_addr learned my addr 172.21.15.16:0/508114197 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72dce5640 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838bb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 -- 172.21.15.16:0/508114197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 msgr2=0x5624838cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 -- 172.21.15.16:0/508114197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 msgr2=0x5624838bb5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838bb5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.729+0000 7ff72ece7640 1 -- 172.21.15.16:0/508114197 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624838b6720 con 0x56248350d350 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.730+0000 7ff72ece7640 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff720014040 tx=0x7ff720003a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=224b0812ced27352 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.730+0000 7ff713fff640 1 -- 172.21.15.16:0/508114197 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff720023040 con 0x56248350d350 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.730+0000 7ff713fff640 1 -- 172.21.15.16:0/508114197 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff720005730 con 0x56248350d350 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.730+0000 7ff713fff640 1 -- 172.21.15.16:0/508114197 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff72002c660 con 0x56248350d350 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- 172.21.15.16:0/508114197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 msgr2=0x5624838b5e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7ff720014040 tx=0x7ff720003a70 comp rx=0 tx=0).stop 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- 172.21.15.16:0/508114197 shutdown_connections 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56248350d350 0x5624838b5e20 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834ec670 0x5624838cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.734 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- 172.21.15.16:0/508114197 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5624834e4490 0x5624838bb5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- 172.21.15.16:0/508114197 >> 172.21.15.16:0/508114197 conn(0x562483501740 msgr2=0x5624838c1330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- 172.21.15.16:0/508114197 shutdown_connections 2024-09-21T03:08:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- 172.21.15.16:0/508114197 wait complete. 2024-09-21T03:08:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- start start 2024-09-21T03:08:10.735 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 Processor -- start 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x56248393bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281ac3d0 con 0x5624834ec670 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281ad840 con 0x56248350d350 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.731+0000 7ff736cbe740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff7281acaa0 con 0x5624834e4490 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff736cbe740 10 client.0 _ll_register_callbacks cb 0x7ff718010930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:10.737 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58584/0 (socket says 172.21.15.16:58584) 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 -- 172.21.15.16:0/4209193040 learned_addr learned my addr 172.21.15.16:0/4209193040 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72ece7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45616/0 (socket says 172.21.15.16:45616) 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72ece7640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x56248393bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 -- 172.21.15.16:0/4209193040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 msgr2=0x5624838c45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 -- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 msgr2=0x56248393bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x56248393bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 -- 172.21.15.16:0/4209193040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56248393c1d0 con 0x5624834ec670 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72ece7640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff72dce5640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff718000c00 tx=0x7ff718009340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476122f2d6f78169 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff711ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:10.738 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff7180037f0 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7ff718003990 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.732+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff720005730 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7ff718003b90 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 10 client.16302 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56248393d490 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff720005730 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7ff71803e070 con 0x5624834ec670 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 10 client.16302 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 10 client.16302 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 20 client.16302 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834a1080 0x56248393fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 4 client.16302 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562483828860 con 0x5624834a1080 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.733+0000 7ff72ece7640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834a1080 0x56248393fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:10.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.734+0000 7ff72ece7640 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834a1080 0x56248393fbf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eecc221f7c7d4ca server_cookie=6cb6af8768a2cb6d in_seq=0 out_seq=0 2024-09-21T03:08:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.734+0000 7ff711ffb640 10 client.16302 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:10.740 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff711ffb640 1 -- 172.21.15.16:0/4209193040 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562483828860 con 0x5624834a1080 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff711ffb640 10 client.16302 handle_command_reply: tid=0 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 client.16302 shutdown 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 msgr2=0x5624838c4b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff718000c00 tx=0x7ff718009340 comp rx=0 tx=0).stop 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 shutdown_connections 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5624834a1080 0x56248393fbf0 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5624834ec670 0x5624838c4b10 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56248350d350 0x56248393bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 --2- 172.21.15.16:0/4209193040 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5624834e4490 0x5624838c45d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 >> 172.21.15.16:0/4209193040 conn(0x562483501740 msgr2=0x5624838c1c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.735+0000 7ff711ffb640 0 client.16302 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 shutdown_connections 2024-09-21T03:08:10.741 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff736cbe740 1 -- 172.21.15.16:0/4209193040 wait complete. 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff736cbe740 20 client.16302 trim_cache size 0 max 16384 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "f18ec32e-7c06-4600-a70e-b062bbda174f", 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:08:10.742 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 msgr2=0x7ff7140b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 0x7ff7140b8c10 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7ff7241052c0 tx=0x7ff724031400 comp rx=0 tx=0).stop 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 msgr2=0x7ff72819a4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7ff724005ab0 tx=0x7ff724003c10 comp rx=0 tx=0).stop 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 shutdown_connections 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff7140b6700 0x7ff7140b8c10 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff7281094b0 0x7ff72819aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff728109e00 0x7ff72819fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.743 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 --2- 172.21.15.16:0/2411391841 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff728103b70 0x7ff72819a4a0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.736+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 >> 172.21.15.16:0/2411391841 conn(0x7ff7280ff5e0 msgr2=0x7ff728101a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.737+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 shutdown_connections 2024-09-21T03:08:10.744 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:10.737+0000 7ff73074a640 1 -- 172.21.15.16:0/2411391841 wait complete. 2024-09-21T03:08:11.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:10 smithi016 ceph-mon[31253]: pgmap v558: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 467 MiB/s wr, 1.07k op/s 2024-09-21T03:08:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:10 smithi018 ceph-mon[34587]: pgmap v558: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 467 MiB/s wr, 1.07k op/s 2024-09-21T03:08:11.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:10 smithi123 ceph-mon[34645]: pgmap v558: 97 pgs: 6 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 87 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 467 MiB/s wr, 1.07k op/s 2024-09-21T03:08:11.819 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:08:12.160 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:11 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:08:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:11 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:11 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:08:12.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:11 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:11 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:08:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:11 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:11 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:08:12.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:11 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:11 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:08:12.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:11 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:11 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:08:12.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:11 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 -- 172.21.15.16:0/62305948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88108cf0 msgr2=0x7fcd8810f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 --2- 172.21.15.16:0/62305948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88108cf0 0x7fcd8810f520 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00d910 tx=0x7fcd7c0329c0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 -- 172.21.15.16:0/62305948 shutdown_connections 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 --2- 172.21.15.16:0/62305948 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd881083a0 0x7fcd881087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 --2- 172.21.15.16:0/62305948 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88108cf0 0x7fcd8810f520 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 --2- 172.21.15.16:0/62305948 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd88102a60 0x7fcd88102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.820+0000 7fcd8e939640 1 -- 172.21.15.16:0/62305948 >> 172.21.15.16:0/62305948 conn(0x7fcd88079730 msgr2=0x7fcd88079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.821+0000 7fcd8e939640 1 -- 172.21.15.16:0/62305948 shutdown_connections 2024-09-21T03:08:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.821+0000 7fcd8e939640 1 -- 172.21.15.16:0/62305948 wait complete. 2024-09-21T03:08:12.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.822+0000 7fcd8e939640 1 Processor -- start 2024-09-21T03:08:12.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.832+0000 7fcd8e939640 1 -- start start 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88102a60 0x7fcd88074a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd881083a0 0x7fcd88074fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8806d2b0 con 0x7fcd88108cf0 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8806d130 con 0x7fcd88102a60 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.835+0000 7fcd8e939640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd8806d430 con 0x7fcd881083a0 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.836+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.836+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58600/0 (socket says 172.21.15.16:58600) 2024-09-21T03:08:12.840 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.836+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2485930653 learned_addr learned my addr 172.21.15.16:0/2485930653 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.837+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2485930653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd881083a0 msgr2=0x7fcd88074fa0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:08:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.837+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd881083a0 0x7fcd88074fa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.837+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2485930653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88102a60 msgr2=0x7fcd88074a20 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:08:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.837+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88102a60 0x7fcd88074a20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.837+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2485930653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd8806f6b0 con 0x7fcd88108cf0 2024-09-21T03:08:12.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.841+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009230 tx=0x7fcd7c005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.846+0000 7fcd857fa640 1 -- 172.21.15.16:0/2485930653 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c014070 con 0x7fcd88108cf0 2024-09-21T03:08:12.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.847+0000 7fcd857fa640 1 -- 172.21.15.16:0/2485930653 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd7c005640 con 0x7fcd88108cf0 2024-09-21T03:08:12.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.848+0000 7fcd857fa640 1 -- 172.21.15.16:0/2485930653 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c003c90 con 0x7fcd88108cf0 2024-09-21T03:08:12.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.852+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd8806f940 con 0x7fcd88108cf0 2024-09-21T03:08:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.852+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd881be360 con 0x7fcd88108cf0 2024-09-21T03:08:12.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.859+0000 7fcd94ead740 1 Processor -- start 2024-09-21T03:08:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.875+0000 7fcd857fa640 1 -- 172.21.15.16:0/2485930653 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcd7c03e070 con 0x7fcd88108cf0 2024-09-21T03:08:12.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.876+0000 7fcd94ead740 1 -- start start 2024-09-21T03:08:12.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.877+0000 7fcd857fa640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 0x7fcd700b8b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.881+0000 7fcd877fe640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 0x7fcd700b8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.883+0000 7fcd857fa640 1 -- 172.21.15.16:0/2485930653 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7fcd7c004b60 con 0x7fcd88108cf0 2024-09-21T03:08:12.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.884+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31d270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.885+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.885+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e31dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.885+0000 7fcd94ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c7e293e50 con 0x556c7df4d1b0 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.885+0000 7fcd94ead740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556c7e283280 con 0x556c7df55390 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.885+0000 7fcd94ead740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c7e287090 con 0x556c7df76070 2024-09-21T03:08:12.888 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.886+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e31dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.887+0000 7fcd877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.887+0000 7fcd877fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58602/0 (socket says 172.21.15.16:58602) 2024-09-21T03:08:12.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.887+0000 7fcd877fe640 1 -- 172.21.15.16:0/911033377 learned_addr learned my addr 172.21.15.16:0/911033377 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:12.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.890+0000 7fcd87fff640 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31d270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.893+0000 7fcd877fe640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 0x7fcd700b8b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcd88107990 tx=0x7fcd7800f040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:12.897 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.895+0000 7fcd877fe640 1 -- 172.21.15.16:0/911033377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 msgr2=0x556c7e31d270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.895+0000 7fcd877fe640 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31d270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.895+0000 7fcd877fe640 1 -- 172.21.15.16:0/911033377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 msgr2=0x556c7e31dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.895+0000 7fcd877fe640 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e31dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.898 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.895+0000 7fcd877fe640 1 -- 172.21.15.16:0/911033377 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7dfe1650 con 0x556c7df4d1b0 2024-09-21T03:08:12.901 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.898+0000 7fcd877fe640 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800d560 tx=0x7fcd78011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4fd3aa9463a2647 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.902+0000 7fcd6e7fc640 1 -- 172.21.15.16:0/911033377 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78014070 con 0x556c7df4d1b0 2024-09-21T03:08:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.903+0000 7fcd6e7fc640 1 -- 172.21.15.16:0/911033377 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd78005020 con 0x556c7df4d1b0 2024-09-21T03:08:12.905 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.903+0000 7fcd6e7fc640 1 -- 172.21.15.16:0/911033377 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd78004c60 con 0x556c7df4d1b0 2024-09-21T03:08:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.904+0000 7fcd94ead740 1 -- 172.21.15.16:0/911033377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 msgr2=0x556c7e31d7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.904+0000 7fcd94ead740 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fcd7800d560 tx=0x7fcd78011000 comp rx=0 tx=0).stop 2024-09-21T03:08:12.907 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.905+0000 7fcd94ead740 1 -- 172.21.15.16:0/911033377 shutdown_connections 2024-09-21T03:08:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.905+0000 7fcd94ead740 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e31d7b0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.905+0000 7fcd94ead740 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e31dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.905+0000 7fcd94ead740 1 --2- 172.21.15.16:0/911033377 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31d270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.908 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.905+0000 7fcd94ead740 1 -- 172.21.15.16:0/911033377 >> 172.21.15.16:0/911033377 conn(0x556c7df6a460 msgr2=0x556c7e31b160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:12.911 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.906+0000 7fcd94ead740 1 -- 172.21.15.16:0/911033377 shutdown_connections 2024-09-21T03:08:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.909+0000 7fcd94ead740 1 -- 172.21.15.16:0/911033377 wait complete. 2024-09-21T03:08:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.909+0000 7fcd94ead740 1 -- start start 2024-09-21T03:08:12.912 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 Processor -- start 2024-09-21T03:08:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e328970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e328eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x556c7e0e4700 con 0x556c7df4d1b0 2024-09-21T03:08:12.913 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x556c7e21e110 con 0x556c7df55390 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x556c7e32ba50 con 0x556c7df76070 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.910+0000 7fcd94ead740 10 client.0 _ll_register_callbacks cb 0x556c7e237990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.912+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.912+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45642/0 (socket says 172.21.15.16:45642) 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.912+0000 7fcd8ced6640 1 -- 172.21.15.16:0/3822606074 learned_addr learned my addr 172.21.15.16:0/3822606074 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:12.914 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.912+0000 7fcd877fe640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e328eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd87fff640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e328970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd8ced6640 1 -- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 msgr2=0x556c7e328eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e328eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.915 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd8ced6640 1 -- 172.21.15.16:0/3822606074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 msgr2=0x556c7e328970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e328970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:12.916 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.913+0000 7fcd8ced6640 1 -- 172.21.15.16:0/3822606074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7e24bc40 con 0x556c7df76070 2024-09-21T03:08:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.914+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0c9210 tx=0x7fcd7c0c7e40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c02e1899a3ecfe7d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:12.917 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.914+0000 7fcd677fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.925+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0f3020 con 0x556c7df76070 2024-09-21T03:08:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.925+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd78005020 con 0x556c7df76070 2024-09-21T03:08:12.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:12.925+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7fcd7c0c7b00 con 0x556c7df76070 2024-09-21T03:08:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.075+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcd7c0cb710 con 0x556c7df76070 2024-09-21T03:08:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.075+0000 7fcd94ead740 10 client.25948 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:13.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.075+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556c7e3377c0 con 0x556c7df76070 2024-09-21T03:08:13.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.076+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd78005020 con 0x556c7df76070 2024-09-21T03:08:13.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fcd7c0b71b0 con 0x556c7df76070 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 10 client.25948 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 10 client.25948 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 20 client.25948 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df349f0 0x556c7e33ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 4 client.25948 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:13.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.078+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c7e295da0 con 0x556c7df349f0 2024-09-21T03:08:13.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.079+0000 7fcd87fff640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df349f0 0x556c7e33ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.081+0000 7fcd87fff640 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df349f0 0x556c7e33ade0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8639a8da4e4d5e9 server_cookie=f2968797a6e98343 in_seq=0 out_seq=0 2024-09-21T03:08:13.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.081+0000 7fcd677fe640 10 client.25948 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.085+0000 7fcd677fe640 1 -- 172.21.15.16:0/3822606074 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x556c7e295da0 con 0x556c7df349f0 2024-09-21T03:08:13.087 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.085+0000 7fcd677fe640 10 client.25948 handle_command_reply: tid=0 2024-09-21T03:08:13.088 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.085+0000 7fcd94ead740 1 client.25948 shutdown 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.087+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 msgr2=0x556c7e334d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.087+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0c9210 tx=0x7fcd7c0c7e40 comp rx=0 tx=0).stop 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 shutdown_connections 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df349f0 0x556c7e33ade0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e334d40 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e328eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 --2- 172.21.15.16:0/3822606074 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df4d1b0 0x556c7e328970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 >> 172.21.15.16:0/3822606074 conn(0x556c7df6a460 msgr2=0x556c7e320400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd677fe640 0 client.25948 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:13.091 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 shutdown_connections 2024-09-21T03:08:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 1 -- 172.21.15.16:0/3822606074 wait complete. 2024-09-21T03:08:13.092 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.088+0000 7fcd94ead740 20 client.25948 trim_cache size 0 max 16384 2024-09-21T03:08:13.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.126+0000 7fcd94ead740 1 Processor -- start 2024-09-21T03:08:13.129 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:12 smithi016 ceph-mon[31253]: pgmap v559: 97 pgs: 25 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 54 active+clean; 70 GiB data, 212 GiB used, 860 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 401 MiB/s wr, 1.18k op/s 2024-09-21T03:08:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.126+0000 7fcd94ead740 1 -- start start 2024-09-21T03:08:13.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df4d1b0 0x556c7e32a5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df55390 0x556c7e31a710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c4c20 con 0x556c7df55390 2024-09-21T03:08:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c73f0 con 0x556c7df4d1b0 2024-09-21T03:08:13.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd94ead740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c51e0 con 0x556c7df76070 2024-09-21T03:08:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45656/0 (socket says 172.21.15.16:45656) 2024-09-21T03:08:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.127+0000 7fcd8ced6640 1 -- 172.21.15.16:0/4037749813 learned_addr learned my addr 172.21.15.16:0/4037749813 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.129+0000 7fcd877fe640 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df55390 0x556c7e31a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.129+0000 7fcd87fff640 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df4d1b0 0x556c7e32a5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.133+0000 7fcd8ced6640 1 -- 172.21.15.16:0/4037749813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df4d1b0 msgr2=0x556c7e32a5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.133+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df4d1b0 0x556c7e32a5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.133+0000 7fcd8ced6640 1 -- 172.21.15.16:0/4037749813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df55390 msgr2=0x556c7e31a710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.133+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df55390 0x556c7e31a710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.133+0000 7fcd8ced6640 1 -- 172.21.15.16:0/4037749813 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7e3235a0 con 0x556c7df76070 2024-09-21T03:08:13.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.137+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00d860 tx=0x7fcd7c0c7f70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=38c594d9844910e7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.140+0000 7fcd6dffb640 1 -- 172.21.15.16:0/4037749813 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0f3070 con 0x556c7df76070 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.140+0000 7fcd6dffb640 1 -- 172.21.15.16:0/4037749813 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd7c0cbab0 con 0x556c7df76070 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.140+0000 7fcd6dffb640 1 -- 172.21.15.16:0/4037749813 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c004dc0 con 0x556c7df76070 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 -- 172.21.15.16:0/4037749813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 msgr2=0x556c7e31ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c00d860 tx=0x7fcd7c0c7f70 comp rx=0 tx=0).stop 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 -- 172.21.15.16:0/4037749813 shutdown_connections 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df55390 0x556c7e31a710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.144 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df76070 0x556c7e31ac50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 --2- 172.21.15.16:0/4037749813 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df4d1b0 0x556c7e32a5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 -- 172.21.15.16:0/4037749813 >> 172.21.15.16:0/4037749813 conn(0x556c7df6a460 msgr2=0x556c7e329e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 -- 172.21.15.16:0/4037749813 shutdown_connections 2024-09-21T03:08:13.145 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.141+0000 7fcd94ead740 1 -- 172.21.15.16:0/4037749813 wait complete. 2024-09-21T03:08:13.148 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.142+0000 7fcd94ead740 1 -- start start 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.142+0000 7fcd94ead740 1 Processor -- start 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df4d1b0 0x556c7e32c700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e32cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c58b0 con 0x556c7df76070 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c6d20 con 0x556c7df55390 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcd881c5f80 con 0x556c7df4d1b0 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.143+0000 7fcd94ead740 10 client.0 _ll_register_callbacks cb 0x7fcd7c032f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.144+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.144+0000 7fcd8ced6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:58638/0 (socket says 172.21.15.16:58638) 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.144+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2326028915 learned_addr learned my addr 172.21.15.16:0/2326028915 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.144+0000 7fcd87fff640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df4d1b0 0x556c7e32c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.145+0000 7fcd877fe640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e32cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2326028915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df4d1b0 msgr2=0x556c7e32c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:13.149 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df4d1b0 0x556c7e32c700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 msgr2=0x556c7e32cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e32cce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 -- 172.21.15.16:0/2326028915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7e39def0 con 0x556c7df76070 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd8ced6640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0d5b00 tx=0x7fcd7c103d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b44353788f388a2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.146+0000 7fcd677fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.147+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd7c0f3040 con 0x556c7df76070 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.147+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcd7c0cbab0 con 0x556c7df76070 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.147+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7fcd7c07b1c0 con 0x556c7df76070 2024-09-21T03:08:13.150 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.148+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fcd7c0e7200 con 0x556c7df76070 2024-09-21T03:08:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.148+0000 7fcd94ead740 10 client.16320 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:13.151 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.148+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x556c7e39def0 con 0x556c7df76070 2024-09-21T03:08:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd7c0cbab0 con 0x556c7df76070 2024-09-21T03:08:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fcd7c0ed030 con 0x556c7df76070 2024-09-21T03:08:13.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 10 client.16320 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 10 client.16320 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 20 client.16320 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df09da0 0x556c7e3a7590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 4 client.16320 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.152+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x556c7e291850 con 0x556c7df09da0 2024-09-21T03:08:13.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.155+0000 7fcd87fff640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df09da0 0x556c7e3a7590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.158+0000 7fcd677fe640 10 client.16320 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:13.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.158+0000 7fcd87fff640 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df09da0 0x556c7e3a7590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b6f5f6bbc05e4f35 server_cookie=d3e709747aa1fe2 in_seq=0 out_seq=0 2024-09-21T03:08:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.158+0000 7fcd677fe640 1 -- 172.21.15.16:0/2326028915 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4546 (crc 0 0 0) 0x556c7e291850 con 0x556c7df09da0 2024-09-21T03:08:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.158+0000 7fcd677fe640 10 client.16320 handle_command_reply: tid=0 2024-09-21T03:08:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.158+0000 7fcd94ead740 1 client.16320 shutdown 2024-09-21T03:08:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.159+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 msgr2=0x556c7e39d880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.159+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c0d5b00 tx=0x7fcd7c103d30 comp rx=0 tx=0).stop 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.160+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 shutdown_connections 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x556c7df09da0 0x556c7e3a7590 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x556c7df76070 0x556c7e39d880 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x556c7df55390 0x556c7e32cce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 --2- 172.21.15.16:0/2326028915 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x556c7df4d1b0 0x556c7e32c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 >> 172.21.15.16:0/2326028915 conn(0x556c7df6a460 msgr2=0x556c7e31ec60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd677fe640 0 client.16320 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 shutdown_connections 2024-09-21T03:08:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.161+0000 7fcd94ead740 1 -- 172.21.15.16:0/2326028915 wait complete. 2024-09-21T03:08:13.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.162+0000 7fcd94ead740 20 client.16320 trim_cache size 0 max 16384 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "f18ec32e-7c06-4600-a70e-b062bbda174f", 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:13.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.170 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:08:13.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:08:13.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:08:13.173 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:13.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.163+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 msgr2=0x7fcd700b8b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 0x7fcd700b8b00 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fcd88107990 tx=0x7fcd7800f040 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 msgr2=0x7fcd8806ee90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fcd7c009230 tx=0x7fcd7c005240 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 shutdown_connections 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcd700b65f0 0x7fcd700b8b00 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcd88108cf0 0x7fcd8806ee90 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcd881083a0 0x7fcd88074fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 --2- 172.21.15.16:0/2485930653 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcd88102a60 0x7fcd88074a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 >> 172.21.15.16:0/2485930653 conn(0x7fcd88079730 msgr2=0x7fcd881051a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.164+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 shutdown_connections 2024-09-21T03:08:13.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:13.166+0000 7fcd8e939640 1 -- 172.21.15.16:0/2485930653 wait complete. 2024-09-21T03:08:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:12 smithi018 ceph-mon[34587]: pgmap v559: 97 pgs: 25 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 54 active+clean; 70 GiB data, 212 GiB used, 860 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 401 MiB/s wr, 1.18k op/s 2024-09-21T03:08:13.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:13 smithi123 ceph-mon[34645]: pgmap v559: 97 pgs: 25 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 54 active+clean; 70 GiB data, 212 GiB used, 860 GiB / 1.0 TiB avail; 1.1 MiB/s rd, 401 MiB/s wr, 1.18k op/s 2024-09-21T03:08:14.189 INFO:tasks.cephfs.filesystem:scrub status for tag:f18ec32e-7c06-4600-a70e-b062bbda174f - {'path': '/', 'tag': 'f18ec32e-7c06-4600-a70e-b062bbda174f', 'options': 'recursive,force'} 2024-09-21T03:08:15.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:14 smithi123 ceph-mon[34645]: pgmap v560: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 1.18k op/s 2024-09-21T03:08:15.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:14 smithi016 ceph-mon[31253]: pgmap v560: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 1.18k op/s 2024-09-21T03:08:15.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:14 smithi018 ceph-mon[34587]: pgmap v560: 97 pgs: 34 active+clean+snaptrim_wait, 22 active+clean+snaptrim, 41 active+clean; 69 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 1.0 MiB/s rd, 362 MiB/s wr, 1.18k op/s 2024-09-21T03:08:17.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:16 smithi123 ceph-mon[34645]: pgmap v561: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 183 KiB/s rd, 247 MiB/s wr, 1.29k op/s 2024-09-21T03:08:17.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:16 smithi016 ceph-mon[31253]: pgmap v561: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 183 KiB/s rd, 247 MiB/s wr, 1.29k op/s 2024-09-21T03:08:17.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:16 smithi018 ceph-mon[34587]: pgmap v561: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 183 KiB/s rd, 247 MiB/s wr, 1.29k op/s 2024-09-21T03:08:19.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:18 smithi123 ceph-mon[34645]: pgmap v562: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 297 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-21T03:08:19.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:18 smithi016 ceph-mon[31253]: pgmap v562: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 297 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-21T03:08:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:18 smithi018 ceph-mon[34587]: pgmap v562: 97 pgs: 38 active+clean+snaptrim_wait, 23 active+clean+snaptrim, 36 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 297 KiB/s rd, 195 MiB/s wr, 1.29k op/s 2024-09-21T03:08:21.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:20 smithi016 ceph-mon[31253]: pgmap v563: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 277 KiB/s rd, 185 MiB/s wr, 1.49k op/s 2024-09-21T03:08:21.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:20 smithi018 ceph-mon[34587]: pgmap v563: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 277 KiB/s rd, 185 MiB/s wr, 1.49k op/s 2024-09-21T03:08:21.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:20 smithi123 ceph-mon[34645]: pgmap v563: 97 pgs: 34 active+clean+snaptrim_wait, 21 active+clean+snaptrim, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 277 KiB/s rd, 185 MiB/s wr, 1.49k op/s 2024-09-21T03:08:23.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:22 smithi016 ceph-mon[31253]: pgmap v564: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 640 KiB/s rd, 197 MiB/s wr, 1.68k op/s 2024-09-21T03:08:23.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:22 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:08:23.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:22 smithi018 ceph-mon[34587]: pgmap v564: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 640 KiB/s rd, 197 MiB/s wr, 1.68k op/s 2024-09-21T03:08:23.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:22 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:08:23.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:22 smithi123 ceph-mon[34645]: pgmap v564: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 640 KiB/s rd, 197 MiB/s wr, 1.68k op/s 2024-09-21T03:08:23.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:22 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:08:25.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:24 smithi016 ceph-mon[31253]: pgmap v565: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 180 MiB/s wr, 1.64k op/s 2024-09-21T03:08:25.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:24 smithi018 ceph-mon[34587]: pgmap v565: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 180 MiB/s wr, 1.64k op/s 2024-09-21T03:08:25.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:24 smithi123 ceph-mon[34645]: pgmap v565: 97 pgs: 34 active+clean+snaptrim_wait, 20 active+clean+snaptrim, 43 active+clean; 66 GiB data, 201 GiB used, 872 GiB / 1.0 TiB avail; 1.2 MiB/s rd, 180 MiB/s wr, 1.64k op/s 2024-09-21T03:08:27.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:26 smithi016 ceph-mon[31253]: pgmap v566: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 233 MiB/s wr, 2.09k op/s 2024-09-21T03:08:27.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:27 smithi018 ceph-mon[34587]: pgmap v566: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 233 MiB/s wr, 2.09k op/s 2024-09-21T03:08:27.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:26 smithi123 ceph-mon[34645]: pgmap v566: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 1.8 MiB/s rd, 233 MiB/s wr, 2.09k op/s 2024-09-21T03:08:29.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:29 smithi016 ceph-mon[31253]: pgmap v567: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 225 MiB/s wr, 2.01k op/s 2024-09-21T03:08:29.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:29 smithi018 ceph-mon[34587]: pgmap v567: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 225 MiB/s wr, 2.01k op/s 2024-09-21T03:08:29.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:29 smithi123 ceph-mon[34645]: pgmap v567: 97 pgs: 32 active+clean+snaptrim_wait, 19 active+clean+snaptrim, 46 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 225 MiB/s wr, 2.01k op/s 2024-09-21T03:08:31.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:31 smithi016 ceph-mon[31253]: pgmap v568: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 259 MiB/s wr, 2.08k op/s 2024-09-21T03:08:31.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:31 smithi018 ceph-mon[34587]: pgmap v568: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 259 MiB/s wr, 2.08k op/s 2024-09-21T03:08:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:31 smithi123 ceph-mon[34645]: pgmap v568: 97 pgs: 32 active+clean+snaptrim_wait, 18 active+clean+snaptrim, 47 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 1.9 MiB/s rd, 259 MiB/s wr, 2.08k op/s 2024-09-21T03:08:32.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:32 smithi016 ceph-mon[31253]: pgmap v569: 97 pgs: 22 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 63 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 309 MiB/s wr, 1.84k op/s 2024-09-21T03:08:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:32 smithi018 ceph-mon[34587]: pgmap v569: 97 pgs: 22 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 63 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 309 MiB/s wr, 1.84k op/s 2024-09-21T03:08:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:32 smithi123 ceph-mon[34645]: pgmap v569: 97 pgs: 22 active+clean+snaptrim_wait, 12 active+clean+snaptrim, 63 active+clean; 65 GiB data, 199 GiB used, 874 GiB / 1.0 TiB avail; 3.2 MiB/s rd, 309 MiB/s wr, 1.84k op/s 2024-09-21T03:08:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:34 smithi018 ceph-mon[34587]: pgmap v570: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 297 MiB/s wr, 1.51k op/s 2024-09-21T03:08:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:34 smithi123 ceph-mon[34645]: pgmap v570: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 297 MiB/s wr, 1.51k op/s 2024-09-21T03:08:35.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:34 smithi016 ceph-mon[31253]: pgmap v570: 97 pgs: 18 active+clean+snaptrim_wait, 11 active+clean+snaptrim, 68 active+clean; 65 GiB data, 199 GiB used, 873 GiB / 1.0 TiB avail; 2.8 MiB/s rd, 297 MiB/s wr, 1.51k op/s 2024-09-21T03:08:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:36 smithi018 ceph-mon[34587]: pgmap v571: 97 pgs: 16 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 74 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 399 MiB/s wr, 1.96k op/s 2024-09-21T03:08:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:36 smithi123 ceph-mon[34645]: pgmap v571: 97 pgs: 16 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 74 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 399 MiB/s wr, 1.96k op/s 2024-09-21T03:08:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:36 smithi016 ceph-mon[31253]: pgmap v571: 97 pgs: 16 active+clean+snaptrim_wait, 7 active+clean+snaptrim, 74 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 399 MiB/s wr, 1.96k op/s 2024-09-21T03:08:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:38 smithi018 ceph-mon[34587]: pgmap v572: 97 pgs: 16 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 75 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 405 MiB/s wr, 1.71k op/s 2024-09-21T03:08:38.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:38 smithi123 ceph-mon[34645]: pgmap v572: 97 pgs: 16 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 75 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 405 MiB/s wr, 1.71k op/s 2024-09-21T03:08:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:38 smithi016 ceph-mon[31253]: pgmap v572: 97 pgs: 16 active+clean+snaptrim_wait, 6 active+clean+snaptrim, 75 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.3 MiB/s rd, 405 MiB/s wr, 1.71k op/s 2024-09-21T03:08:40.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:40 smithi123 ceph-mon[34645]: pgmap v573: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 471 MiB/s wr, 1.59k op/s 2024-09-21T03:08:41.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:40 smithi016 ceph-mon[31253]: pgmap v573: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 471 MiB/s wr, 1.59k op/s 2024-09-21T03:08:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:40 smithi018 ceph-mon[34587]: pgmap v573: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 204 GiB used, 869 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 471 MiB/s wr, 1.59k op/s 2024-09-21T03:08:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:42 smithi123 ceph-mon[34645]: pgmap v574: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 519 MiB/s wr, 1.44k op/s 2024-09-21T03:08:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:42.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:08:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:42 smithi016 ceph-mon[31253]: pgmap v574: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 519 MiB/s wr, 1.44k op/s 2024-09-21T03:08:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:08:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:42 smithi018 ceph-mon[34587]: pgmap v574: 97 pgs: 10 active+clean+snaptrim_wait, 4 active+clean+snaptrim, 83 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 519 MiB/s wr, 1.44k op/s 2024-09-21T03:08:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:08:43.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:08:43.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:08:43.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:08:44.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:08:44.615 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:44 smithi123 ceph-mon[34645]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 467 MiB/s wr, 1.34k op/s 2024-09-21T03:08:45.017 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:44 smithi016 ceph-mon[31253]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 467 MiB/s wr, 1.34k op/s 2024-09-21T03:08:45.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:44 smithi018 ceph-mon[34587]: pgmap v575: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 207 GiB used, 866 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 467 MiB/s wr, 1.34k op/s 2024-09-21T03:08:45.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 -- 172.21.15.16:0/2645371975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408108d20 msgr2=0x7f840810f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 --2- 172.21.15.16:0/2645371975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408108d20 0x7f840810f550 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f840002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 -- 172.21.15.16:0/2645371975 shutdown_connections 2024-09-21T03:08:45.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 --2- 172.21.15.16:0/2645371975 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408108d20 0x7f840810f550 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.483 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 --2- 172.21.15.16:0/2645371975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f84081087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 --2- 172.21.15.16:0/2645371975 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408102ae0 0x7f8408102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 -- 172.21.15.16:0/2645371975 >> 172.21.15.16:0/2645371975 conn(0x7f8408079790 msgr2=0x7f8408079ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 -- 172.21.15.16:0/2645371975 shutdown_connections 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.367+0000 7f840d027640 1 -- 172.21.15.16:0/2645371975 wait complete. 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.368+0000 7f840d027640 1 Processor -- start 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.368+0000 7f840d027640 1 -- start start 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f840819b710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 0x7f840819bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.484 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84081129e0 con 0x7f8408102ae0 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112860 con 0x7f8408108d20 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840d027640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f8408112b60 con 0x7f84081083d0 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f840819b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34642/0 (socket says 172.21.15.16:34642) 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 -- 172.21.15.16:0/2638764378 learned_addr learned my addr 172.21.15.16:0/2638764378 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840759e640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 0x7f840819bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 -- 172.21.15.16:0/2638764378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 msgr2=0x7f840819b710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f840819b710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 -- 172.21.15.16:0/2638764378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 msgr2=0x7f840819bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 0x7f840819bc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f8406d9d640 1 -- 172.21.15.16:0/2638764378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84081a64e0 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840759e640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 0x7f840819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.369+0000 7f840659c640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f840819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f83f800e9f0 tx=0x7f83f800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f83effff640 1 -- 172.21.15.16:0/2638764378 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f800cd80 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84081a67d0 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f83effff640 1 -- 172.21.15.16:0/2638764378 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83f80108e0 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84081a6d60 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.370+0000 7f83effff640 1 -- 172.21.15.16:0/2638764378 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f8018980 con 0x7f8408102ae0 2024-09-21T03:08:45.486 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.372+0000 7f841359b740 1 Processor -- start 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 -- start start 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc51270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc517b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc51cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de1cbc7e50 con 0x55de1c8aa070 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55de1cbb7280 con 0x55de1c8811b0 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f841359b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55de1cbbb090 con 0x55de1c889390 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc51270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc51cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc51270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34656/0 (socket says 172.21.15.16:34656) 2024-09-21T03:08:45.487 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840759e640 1 -- 172.21.15.16:0/1869105808 learned_addr learned my addr 172.21.15.16:0/1869105808 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f83effff640 1 -- 172.21.15.16:0/2638764378 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f83f8010430 con 0x7f8408102ae0 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f8406d9d640 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc517b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 -- 172.21.15.16:0/1869105808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 msgr2=0x55de1cc517b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc517b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 -- 172.21.15.16:0/1869105808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 msgr2=0x55de1cc51270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc51270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.373+0000 7f840659c640 1 -- 172.21.15.16:0/1869105808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de1c915650 con 0x55de1c889390 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f8406d9d640 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f840659c640 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc51cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83fc005fd0 tx=0x7f83fc010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4c8737c8d0a35033 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.488 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f83ed7fa640 1 -- 172.21.15.16:0/1869105808 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc011700 con 0x55de1c889390 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f83ed7fa640 1 -- 172.21.15.16:0/1869105808 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc011d50 con 0x55de1c889390 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f83ed7fa640 1 -- 172.21.15.16:0/1869105808 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83fc01ce20 con 0x55de1c889390 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.374+0000 7f83effff640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 0x7f83f00b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f840759e640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 0x7f83f00b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f83effff640 1 -- 172.21.15.16:0/2638764378 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f83f80e7ab0 con 0x7f8408102ae0 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 -- 172.21.15.16:0/1869105808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 msgr2=0x55de1cc51cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc51cf0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83fc005fd0 tx=0x7f83fc010860 comp rx=0 tx=0).stop 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 -- 172.21.15.16:0/1869105808 shutdown_connections 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc51cf0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.489 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc517b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 --2- 172.21.15.16:0/1869105808 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc51270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 -- 172.21.15.16:0/1869105808 >> 172.21.15.16:0/1869105808 conn(0x55de1c89e460 msgr2=0x55de1cc4f160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f840759e640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 0x7f83f00b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f84000093e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 -- 172.21.15.16:0/1869105808 shutdown_connections 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.375+0000 7f841359b740 1 -- 172.21.15.16:0/1869105808 wait complete. 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 -- start start 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 Processor -- start 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc5cda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.490 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 0x55de1cc5d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55de1ca18700 con 0x55de1c8811b0 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:08:45.491 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.492 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:08:45.493 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.494 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:08:45.495 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.496 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.497 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.498 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.499 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.500 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:45.501 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:45.502 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:45.503 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55de1cb52110 con 0x55de1c8aa070 2024-09-21T03:08:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55de1cc5fa50 con 0x55de1c889390 2024-09-21T03:08:45.503 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f841359b740 10 client.0 _ll_register_callbacks cb 0x55de1cb6b990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34670/0 (socket says 172.21.15.16:34670) 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f8406d9d640 1 -- 172.21.15.16:0/3633485229 learned_addr learned my addr 172.21.15.16:0/3633485229 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f840759e640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 0x55de1cc5d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.376+0000 7f840659c640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc5cda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 -- 172.21.15.16:0/3633485229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 msgr2=0x55de1cc5cda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc5cda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 -- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 msgr2=0x55de1cc5d2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.504 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 0x55de1cc5d2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 -- 172.21.15.16:0/3633485229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1cb7fc40 con 0x55de1c8811b0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f840759e640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 0x55de1cc5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f8406d9d640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f83f8000c00 tx=0x7f83f80ad6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22235c82f6902b89 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f83eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80a9080 con 0x55de1c8811b0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f83f80a9260 con 0x55de1c8811b0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f83fc011d50 con 0x55de1c8811b0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f83f8010720 con 0x55de1c8811b0 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f841359b740 10 client.16332 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:45.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55de1cc6b660 con 0x55de1c8811b0 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.377+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83fc011d50 con 0x55de1c8811b0 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f83f80a9460 con 0x55de1c8811b0 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 10 client.16332 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 10 client.16332 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 20 client.16332 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c8689f0 0x55de1cc6eb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 4 client.16332 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de1cbc9da0 con 0x55de1c8689f0 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f840659c640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c8689f0 0x55de1cc6eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f83eaff5640 10 client.16332 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.378+0000 7f840659c640 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c8689f0 0x55de1cc6eb90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2aa21f0d48c2567 server_cookie=6e3f4a20413b8694 in_seq=0 out_seq=0 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f83eaff5640 1 -- 172.21.15.16:0/3633485229 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55de1cbc9da0 con 0x55de1c8689f0 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f83eaff5640 10 client.16332 handle_command_reply: tid=0 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 client.16332 shutdown 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 msgr2=0x55de1cc5c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f83f8000c00 tx=0x7f83f80ad6e0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 shutdown_connections 2024-09-21T03:08:45.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c8689f0 0x55de1cc6eb90 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8aa070 0x55de1cc5d2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc5cda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f83eaff5640 0 client.16332 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 --2- 172.21.15.16:0/3633485229 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8811b0 0x55de1cc5c860 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 >> 172.21.15.16:0/3633485229 conn(0x55de1c89e460 msgr2=0x55de1cc54450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 shutdown_connections 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 1 -- 172.21.15.16:0/3633485229 wait complete. 2024-09-21T03:08:45.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.379+0000 7f841359b740 20 client.16332 trim_cache size 0 max 16384 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.397+0000 7f841359b740 1 Processor -- start 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.397+0000 7f841359b740 1 -- start start 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc5e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc529b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84081ad7f0 con 0x55de1c8aa070 2024-09-21T03:08:45.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f84081b0050 con 0x55de1c8811b0 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f841359b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84081ade40 con 0x55de1c889390 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34680/0 (socket says 172.21.15.16:34680) 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 -- 172.21.15.16:0/3898454325 learned_addr learned my addr 172.21.15.16:0/3898454325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f8406d9d640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc5e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840659c640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc529b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 -- 172.21.15.16:0/3898454325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 msgr2=0x55de1cc529b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc529b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 -- 172.21.15.16:0/3898454325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 msgr2=0x55de1cc5e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc5e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 -- 172.21.15.16:0/3898454325 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de1cc5ee40 con 0x55de1c8aa070 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f8406d9d640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.398+0000 7f840759e640 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8400009680 tx=0x7f8400045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b08e80fbf54568b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.399+0000 7f83ecff9640 1 -- 172.21.15.16:0/3898454325 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000041c0 con 0x55de1c8aa070 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.399+0000 7f83ecff9640 1 -- 172.21.15.16:0/3898454325 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400036040 con 0x55de1c8aa070 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.399+0000 7f83ecff9640 1 -- 172.21.15.16:0/3898454325 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84000305c0 con 0x55de1c8aa070 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- 172.21.15.16:0/3898454325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 msgr2=0x55de1cc4e710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f8400009680 tx=0x7f8400045000 comp rx=0 tx=0).stop 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- 172.21.15.16:0/3898454325 shutdown_connections 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc4e710 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc529b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- 172.21.15.16:0/3898454325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc5e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- 172.21.15.16:0/3898454325 >> 172.21.15.16:0/3898454325 conn(0x55de1c89e460 msgr2=0x55de1cc522a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- 172.21.15.16:0/3898454325 shutdown_connections 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- 172.21.15.16:0/3898454325 wait complete. 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 -- start start 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 Processor -- start 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc65660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.400+0000 7f841359b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc60bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f841359b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f841359b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f84081ae510 con 0x55de1c8aa070 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f841359b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f84081af980 con 0x55de1c8811b0 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f841359b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f84081aebe0 con 0x55de1c889390 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f840759e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34686/0 (socket says 172.21.15.16:34686) 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc65660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f840759e640 1 -- 172.21.15.16:0/2131564856 learned_addr learned my addr 172.21.15.16:0/2131564856 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f841359b740 10 client.0 _ll_register_callbacks cb 0x7f83f80aa010 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f840659c640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc60bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 -- 172.21.15.16:0/2131564856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 msgr2=0x55de1cc60bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc60bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 -- 172.21.15.16:0/2131564856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 msgr2=0x55de1cc612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 -- 172.21.15.16:0/2131564856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de1ccd8550 con 0x55de1c8811b0 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f840759e640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f8406d9d640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc65660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83f80b3550 tx=0x7f83f80b35d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6fd19241906d8660 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:45.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.401+0000 7f83eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f83f80ad420 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f83f80a9260 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8400036040 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f83f80a9040 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 10 client.25982 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55de1ccd9620 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8400036040 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f83f80b69a0 con 0x55de1c8811b0 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 10 client.25982 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 10 client.25982 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 20 client.25982 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:45.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c83dda0 0x55de1ccdbd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 4 client.25982 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55de1cbc5850 con 0x55de1c83dda0 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.402+0000 7f840659c640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c83dda0 0x55de1ccdbd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.403+0000 7f83eaff5640 10 client.25982 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.403+0000 7f840659c640 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c83dda0 0x55de1ccdbd90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c3155e4c538164d server_cookie=7b6c303189f109a7 in_seq=0 out_seq=0 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f83eaff5640 1 -- 172.21.15.16:0/2131564856 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4350 (crc 0 0 0) 0x55de1cbc5850 con 0x55de1c83dda0 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f83eaff5640 10 client.25982 handle_command_reply: tid=0 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 client.25982 shutdown 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 msgr2=0x55de1cc65660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc65660 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f83f80b3550 tx=0x7f83f80b35d0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 shutdown_connections 2024-09-21T03:08:45.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55de1c83dda0 0x55de1ccdbd90 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55de1c8aa070 0x55de1cc612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55de1c889390 0x55de1cc60bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 --2- 172.21.15.16:0/2131564856 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55de1c8811b0 0x55de1cc65660 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f83eaff5640 0 client.25982 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 >> 172.21.15.16:0/2131564856 conn(0x55de1c89e460 msgr2=0x55de1cc52c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 shutdown_connections 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 1 -- 172.21.15.16:0/2131564856 wait complete. 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f841359b740 20 client.25982 trim_cache size 0 max 16384 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 msgr2=0x7f83f00b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 0x7f83f00b8c10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f8400009ca0 tx=0x7f84000093e0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 msgr2=0x7f840819b1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.404+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f83f800e9f0 tx=0x7f83f800eeb0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 shutdown_connections 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f83f00b6700 0x7f83f00b8c10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f8408108d20 0x7f840819bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84081083d0 0x7f840819b710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 --2- 172.21.15.16:0/2638764378 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f8408102ae0 0x7f840819b1d0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 >> 172.21.15.16:0/2638764378 conn(0x7f8408079790 msgr2=0x7f84081063c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 shutdown_connections 2024-09-21T03:08:45.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:45.405+0000 7f840d027640 1 -- 172.21.15.16:0/2638764378 wait complete. 2024-09-21T03:08:46.319 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:08:46.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:08:46.778 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:46.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:46 smithi123 ceph-mon[34645]: pgmap v576: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 539 MiB/s wr, 1.49k op/s 2024-09-21T03:08:47.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:46 smithi016 ceph-mon[31253]: pgmap v576: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 539 MiB/s wr, 1.49k op/s 2024-09-21T03:08:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:46 smithi018 ceph-mon[34587]: pgmap v576: 97 pgs: 5 active+clean+snaptrim_wait, 3 active+clean+snaptrim, 89 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 539 MiB/s wr, 1.49k op/s 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 -- 172.21.15.16:0/3094719439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec1083a0 msgr2=0x7f15ec1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3094719439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec1083a0 0x7f15ec1087b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f15e0009900 tx=0x7f15e002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 -- 172.21.15.16:0/3094719439 shutdown_connections 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3094719439 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec1083a0 0x7f15ec1087b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3094719439 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 0x7f15ec10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3094719439 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec102a60 0x7f15ec102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 -- 172.21.15.16:0/3094719439 >> 172.21.15.16:0/3094719439 conn(0x7f15ec079730 msgr2=0x7f15ec079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 -- 172.21.15.16:0/3094719439 shutdown_connections 2024-09-21T03:08:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.534+0000 7f15f24ce640 1 -- 172.21.15.16:0/3094719439 wait complete. 2024-09-21T03:08:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.535+0000 7f15f24ce640 1 Processor -- start 2024-09-21T03:08:47.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.536+0000 7f15f24ce640 1 -- start start 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.536+0000 7f15f24ce640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 0x7f15ec19a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.536+0000 7f15f24ce640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 0x7f15ec19ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.536+0000 7f15f24ce640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 0x7f15ec19fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15f24ce640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec1129a0 con 0x7f15ec102a60 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15f24ce640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec112820 con 0x7f15ec108cf0 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15eb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 0x7f15ec19ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15eb7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 0x7f15ec19ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44336/0 (socket says 172.21.15.16:44336) 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15eb7fe640 1 -- 172.21.15.16:0/3215145812 learned_addr learned my addr 172.21.15.16:0/3215145812 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15ebfff640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 0x7f15ec19a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f15ec112b20 con 0x7f15ec1083a0 2024-09-21T03:08:47.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15f0a6b640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 0x7f15ec19fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15ebfff640 1 -- 172.21.15.16:0/3215145812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 msgr2=0x7f15ec19ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15ebfff640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 0x7f15ec19ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15ebfff640 1 -- 172.21.15.16:0/3215145812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 msgr2=0x7f15ec19fdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.537+0000 7f15ebfff640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 0x7f15ec19fdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.538+0000 7f15ebfff640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15ec1a04f0 con 0x7f15ec102a60 2024-09-21T03:08:47.540 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.538+0000 7f15ebfff640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 0x7f15ec19a580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f10 tx=0x7f15dc00ed10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.538+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc020070 con 0x7f15ec102a60 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.539+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15dc00ccc0 con 0x7f15ec102a60 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.539+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15dc01ba80 con 0x7f15ec102a60 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.539+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15ec1a07e0 con 0x7f15ec102a60 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.539+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15ec0ff550 con 0x7f15ec102a60 2024-09-21T03:08:47.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.540+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f15ec102e70 con 0x7f15ec102a60 2024-09-21T03:08:47.544 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.542+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f15dc023070 con 0x7f15ec102a60 2024-09-21T03:08:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.543+0000 7f15e97fa640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 0x7f15d40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.543+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f15dc0eba60 con 0x7f15ec102a60 2024-09-21T03:08:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.543+0000 7f15eb7fe640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 0x7f15d40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:47.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.544+0000 7f15eb7fe640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 0x7f15d40b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15e0005f70 tx=0x7f15e0009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:47.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.549+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f15dc0b62f0 con 0x7f15ec102a60 2024-09-21T03:08:47.928 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.926+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f15ec19b6b0 con 0x7f15ec102a60 2024-09-21T03:08:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.927+0000 7f15e97fa640 1 -- 172.21.15.16:0/3215145812 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f15dc0bb230 con 0x7f15ec102a60 2024-09-21T03:08:47.929 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:08:47.929 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:08:47.929 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 msgr2=0x7f15d40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 0x7f15d40b8c10 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f15e0005f70 tx=0x7f15e0009560 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 msgr2=0x7f15ec19a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 0x7f15ec19a580 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f15dc009f10 tx=0x7f15dc00ed10 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 shutdown_connections 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f15d40b6700 0x7f15d40b8c10 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f15ec108cf0 0x7f15ec19fdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f15ec1083a0 0x7f15ec19ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 --2- 172.21.15.16:0/3215145812 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f15ec102a60 0x7f15ec19a580 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:47.932 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.930+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 >> 172.21.15.16:0/3215145812 conn(0x7f15ec079730 msgr2=0x7f15ec06b290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.931+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 shutdown_connections 2024-09-21T03:08:47.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:47.931+0000 7f15f24ce640 1 -- 172.21.15.16:0/3215145812 wait complete. 2024-09-21T03:08:48.956 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:08:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:48 smithi016 ceph-mon[31253]: pgmap v577: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 455 MiB/s wr, 972 op/s 2024-09-21T03:08:49.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:48 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3215145812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:48 smithi018 ceph-mon[34587]: pgmap v577: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 455 MiB/s wr, 972 op/s 2024-09-21T03:08:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:48 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3215145812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:48 smithi123 ceph-mon[34645]: pgmap v577: 97 pgs: 2 active+clean+snaptrim, 95 active+clean; 68 GiB data, 209 GiB used, 864 GiB / 1.0 TiB avail; 2.7 MiB/s rd, 455 MiB/s wr, 972 op/s 2024-09-21T03:08:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:48 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3215145812' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:08:49.449 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.196+0000 7f32b1140640 1 -- 172.21.15.16:0/2077977318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 msgr2=0x7f32ac1087b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.196+0000 7f32b1140640 1 --2- 172.21.15.16:0/2077977318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac1087b0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32a0009900 tx=0x7f32a002a410 comp rx=0 tx=0).stop 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.197+0000 7f32b1140640 1 -- 172.21.15.16:0/2077977318 shutdown_connections 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.197+0000 7f32b1140640 1 --2- 172.21.15.16:0/2077977318 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac10f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.197+0000 7f32b1140640 1 --2- 172.21.15.16:0/2077977318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac1087b0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.197+0000 7f32b1140640 1 --2- 172.21.15.16:0/2077977318 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ac102a60 0x7f32ac102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.197+0000 7f32b1140640 1 -- 172.21.15.16:0/2077977318 >> 172.21.15.16:0/2077977318 conn(0x7f32ac079730 msgr2=0x7f32ac079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.198+0000 7f32b1140640 1 -- 172.21.15.16:0/2077977318 shutdown_connections 2024-09-21T03:08:50.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.198+0000 7f32b1140640 1 -- 172.21.15.16:0/2077977318 wait complete. 2024-09-21T03:08:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.200+0000 7f32b1140640 1 Processor -- start 2024-09-21T03:08:50.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.200+0000 7f32b1140640 1 -- start start 2024-09-21T03:08:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ac102a60 0x7f32ac199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac19ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112a70 con 0x7f32ac108cf0 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac1128f0 con 0x7f32ac102a60 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32b1140640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac112bf0 con 0x7f32ac1083a0 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.201+0000 7f32aa59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.202+0000 7f32aa59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac199a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44356/0 (socket says 172.21.15.16:44356) 2024-09-21T03:08:50.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.202+0000 7f32aa59c640 1 -- 172.21.15.16:0/1223426753 learned_addr learned my addr 172.21.15.16:0/1223426753 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:50.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.202+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac19ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.203+0000 7f32ab59e640 1 -- 172.21.15.16:0/1223426753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 msgr2=0x7f32ac199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.203+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.203+0000 7f32ab59e640 1 -- 172.21.15.16:0/1223426753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ac102a60 msgr2=0x7f32ac199480 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.203+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ac102a60 0x7f32ac199480 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.203+0000 7f32ab59e640 1 -- 172.21.15.16:0/1223426753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32ac19f3f0 con 0x7f32ac108cf0 2024-09-21T03:08:50.206 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.204+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac19ecc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f329c00db00 tx=0x7f329c00e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.205+0000 7f3293fff640 1 -- 172.21.15.16:0/1223426753 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c00e8f0 con 0x7f32ac108cf0 2024-09-21T03:08:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.205+0000 7f3293fff640 1 -- 172.21.15.16:0/1223426753 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c0094c0 con 0x7f32ac108cf0 2024-09-21T03:08:50.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.205+0000 7f3293fff640 1 -- 172.21.15.16:0/1223426753 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c016650 con 0x7f32ac108cf0 2024-09-21T03:08:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.206+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32ac19f680 con 0x7f32ac108cf0 2024-09-21T03:08:50.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.206+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32ac06ae60 con 0x7f32ac108cf0 2024-09-21T03:08:50.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.212+0000 7f32b76b4740 1 Processor -- start 2024-09-21T03:08:50.218 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.216+0000 7f3293fff640 1 -- 172.21.15.16:0/1223426753 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f329c009670 con 0x7f32ac108cf0 2024-09-21T03:08:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.218+0000 7f3293fff640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 0x7f32940b8d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.219+0000 7f3293fff640 1 -- 172.21.15.16:0/1223426753 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f329c0e69f0 con 0x7f32ac108cf0 2024-09-21T03:08:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 -- start start 2024-09-21T03:08:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d378129cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 0x55d37812a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d64390 0x55d37813c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d3780a2ea0 con 0x55d377d85070 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d3780922d0 con 0x55d377d5c1b0 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.220+0000 7f32b76b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d3780960e0 con 0x55d377d64390 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.221+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 0x7f32940b8d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.221+0000 7f32aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d378129cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.221+0000 7f32aad9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d378129cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34748/0 (socket says 172.21.15.16:34748) 2024-09-21T03:08:50.223 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.221+0000 7f32aad9d640 1 -- 172.21.15.16:0/274521047 learned_addr learned my addr 172.21.15.16:0/274521047 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:50.225 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.223+0000 7f32aa59c640 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 0x55d37812a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.224+0000 7f32aa59c640 1 -- 172.21.15.16:0/274521047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d64390 msgr2=0x55d37813c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.224+0000 7f32aa59c640 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d64390 0x55d37813c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.224+0000 7f32aa59c640 1 -- 172.21.15.16:0/274521047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 msgr2=0x55d378129cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.224+0000 7f32aa59c640 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d378129cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.226 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.224+0000 7f32aa59c640 1 -- 172.21.15.16:0/274521047 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d377df0650 con 0x55d377d5c1b0 2024-09-21T03:08:50.227 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.225+0000 7f32aa59c640 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 0x55d37812a270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32a0006010 tx=0x7f32a0044000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=24d2232facc0234c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.226+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 0x7f32940b8d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32ac107b40 tx=0x7f32a0003720 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.226+0000 7f32917fa640 1 -- 172.21.15.16:0/274521047 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a0035040 con 0x55d377d5c1b0 2024-09-21T03:08:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.227+0000 7f32917fa640 1 -- 172.21.15.16:0/274521047 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a004f070 con 0x55d377d5c1b0 2024-09-21T03:08:50.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32917fa640 1 -- 172.21.15.16:0/274521047 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a004a6f0 con 0x55d377d5c1b0 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 -- 172.21.15.16:0/274521047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 msgr2=0x55d37812a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 0x55d37812a270 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f32a0006010 tx=0x7f32a0044000 comp rx=0 tx=0).stop 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 -- 172.21.15.16:0/274521047 shutdown_connections 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d64390 0x55d37813c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d5c1b0 0x55d37812a270 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 --2- 172.21.15.16:0/274521047 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d378129cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 -- 172.21.15.16:0/274521047 >> 172.21.15.16:0/274521047 conn(0x55d377d79460 msgr2=0x55d37812c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.228+0000 7f32b76b4740 1 -- 172.21.15.16:0/274521047 shutdown_connections 2024-09-21T03:08:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.232+0000 7f32b76b4740 1 -- 172.21.15.16:0/274521047 wait complete. 2024-09-21T03:08:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.232+0000 7f32b76b4740 1 -- start start 2024-09-21T03:08:50.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.232+0000 7f32b76b4740 1 Processor -- start 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d5c1b0 0x55d378140970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d378140f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d377ef3700 con 0x55d377d5c1b0 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d37802d160 con 0x55d377d64390 2024-09-21T03:08:50.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d37813d130 con 0x55d377d85070 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.233+0000 7f32b76b4740 10 client.0 _ll_register_callbacks cb 0x55d3780469e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.234+0000 7f32ab59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.234+0000 7f32ab59e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44372/0 (socket says 172.21.15.16:44372) 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.234+0000 7f32ab59e640 1 -- 172.21.15.16:0/1361764330 learned_addr learned my addr 172.21.15.16:0/1361764330 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.234+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d378140f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.234+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d5c1b0 0x55d378140970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.235+0000 7f32ab59e640 1 -- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 msgr2=0x55d378140f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.235+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d378140f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.235+0000 7f32ab59e640 1 -- 172.21.15.16:0/1361764330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d5c1b0 msgr2=0x55d378140970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.235+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d5c1b0 0x55d378140970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.235+0000 7f32ab59e640 1 -- 172.21.15.16:0/1361764330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d37805ac90 con 0x55d377d85070 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.236+0000 7f32ab59e640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ac4a0 tx=0x7f329c0ac0f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c26672739e3187c8 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.236+0000 7f328eff5640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.236+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0acea0 con 0x55d377d85070 2024-09-21T03:08:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.236+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f32a004f070 con 0x55d377d85070 2024-09-21T03:08:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.236+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f329c00eca0 con 0x55d377d85070 2024-09-21T03:08:50.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.237+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f329c0b1280 con 0x55d377d85070 2024-09-21T03:08:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.238+0000 7f32b76b4740 10 client.25984 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:50.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.238+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d37814ef10 con 0x55d377d85070 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32a004f070 con 0x55d377d85070 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f329c0adad0 con 0x55d377d85070 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 10 client.25984 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 10 client.25984 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 20 client.25984 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d439f0 0x55d378152730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.242 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 4 client.25984 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.239+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3780a4df0 con 0x55d377d439f0 2024-09-21T03:08:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.240+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d439f0 0x55d378152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.241+0000 7f328eff5640 10 client.25984 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:50.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.242+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d439f0 0x55d378152730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ee3111737620ec59 server_cookie=61860b9048abb3e7 in_seq=0 out_seq=0 2024-09-21T03:08:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.244+0000 7f328eff5640 1 -- 172.21.15.16:0/1361764330 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d3780a4df0 con 0x55d377d439f0 2024-09-21T03:08:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.244+0000 7f328eff5640 10 client.25984 handle_command_reply: tid=0 2024-09-21T03:08:50.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 client.25984 shutdown 2024-09-21T03:08:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 msgr2=0x55d378145f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f329c0ac4a0 tx=0x7f329c0ac0f0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 shutdown_connections 2024-09-21T03:08:50.247 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d439f0 0x55d378152730 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d85070 0x55d378145f70 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d378140f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1361764330 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d5c1b0 0x55d378140970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 >> 172.21.15.16:0/1361764330 conn(0x55d377d79460 msgr2=0x55d3781290e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f328eff5640 0 client.25984 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 shutdown_connections 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.245+0000 7f32b76b4740 1 -- 172.21.15.16:0/1361764330 wait complete. 2024-09-21T03:08:50.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.246+0000 7f32b76b4740 20 client.25984 trim_cache size 0 max 16384 2024-09-21T03:08:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.269+0000 7f32b76b4740 1 Processor -- start 2024-09-21T03:08:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.269+0000 7f32b76b4740 1 -- start start 2024-09-21T03:08:50.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d37813a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d37812c0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac072670 con 0x55d377d85070 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac074ed0 con 0x55d377d64390 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32b76b4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac072cc0 con 0x55d377d5c1b0 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32ab59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32ab59e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34758/0 (socket says 172.21.15.16:34758) 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.270+0000 7f32ab59e640 1 -- 172.21.15.16:0/3795248569 learned_addr learned my addr 172.21.15.16:0/3795248569 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 -- 172.21.15.16:0/3795248569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 msgr2=0x55d37813a3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d37813a3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 -- 172.21.15.16:0/3795248569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 msgr2=0x55d37812c0f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d37812c0f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 -- 172.21.15.16:0/3795248569 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d37812cb00 con 0x55d377d85070 2024-09-21T03:08:50.273 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f32ab59e640 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e8c0 tx=0x7f329c00bae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24e7dd41513ee0f1 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f3290ff9640 1 -- 172.21.15.16:0/3795248569 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0acea0 con 0x55d377d85070 2024-09-21T03:08:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f3290ff9640 1 -- 172.21.15.16:0/3795248569 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c0c6070 con 0x55d377d85070 2024-09-21T03:08:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.271+0000 7f3290ff9640 1 -- 172.21.15.16:0/3795248569 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f329c0ab6d0 con 0x55d377d85070 2024-09-21T03:08:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 -- 172.21.15.16:0/3795248569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 msgr2=0x55d37813b0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.275 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f329c00e8c0 tx=0x7f329c00bae0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 -- 172.21.15.16:0/3795248569 shutdown_connections 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d85070 0x55d37813b0e0 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d64390 0x55d37812c0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 --2- 172.21.15.16:0/3795248569 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d37813a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 -- 172.21.15.16:0/3795248569 >> 172.21.15.16:0/3795248569 conn(0x55d377d79460 msgr2=0x55d378129c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 -- 172.21.15.16:0/3795248569 shutdown_connections 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.273+0000 7f32b76b4740 1 -- 172.21.15.16:0/3795248569 wait complete. 2024-09-21T03:08:50.276 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.274+0000 7f32b76b4740 1 -- start start 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.274+0000 7f32b76b4740 1 Processor -- start 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d378158880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d85070 0x55d3781476a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32aa59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34774/0 (socket says 172.21.15.16:34774) 2024-09-21T03:08:50.278 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32aa59c640 1 -- 172.21.15.16:0/1155730363 learned_addr learned my addr 172.21.15.16:0/1155730363 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac073390 con 0x55d377d64390 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac074800 con 0x55d377d85070 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f32ac073a60 con 0x55d377d5c1b0 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d378158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.275+0000 7f32b76b4740 10 client.0 _ll_register_callbacks cb 0x7f329c0a8660 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aa59c640 1 -- 172.21.15.16:0/1155730363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 msgr2=0x55d378158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d378158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aa59c640 1 -- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d85070 msgr2=0x55d3781476a0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d85070 0x55d3781476a0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aa59c640 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d37814bfa0 con 0x55d377d64390 2024-09-21T03:08:50.279 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.276+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d378158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f32aa59c640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32a004b000 tx=0x7f32a002a9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14bde2c729efb973 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:50.280 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f328eff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32a00314a0 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f32a0031640 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f329c0c6070 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f32a0031840 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f32b76b4740 10 client.16356 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.277+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d3781bd200 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f329c0c6070 con 0x55d377d64390 2024-09-21T03:08:50.281 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f32a0063030 con 0x55d377d64390 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 10 client.16356 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 10 client.16356 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 20 client.16356 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d18da0 0x55d3781bf4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 4 client.16356 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.278+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d3780a08a0 con 0x55d377d18da0 2024-09-21T03:08:50.282 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.280+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d18da0 0x55d3781bf4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.283+0000 7f32aad9d640 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d18da0 0x55d3781bf4f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38342dc5f2824a2f server_cookie=79d0372f426f076d in_seq=0 out_seq=0 2024-09-21T03:08:50.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.283+0000 7f328eff5640 10 client.16356 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.284+0000 7f328eff5640 1 -- 172.21.15.16:0/1155730363 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d3780a08a0 con 0x55d377d18da0 2024-09-21T03:08:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.284+0000 7f328eff5640 10 client.16356 handle_command_reply: tid=0 2024-09-21T03:08:50.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.284+0000 7f32b76b4740 1 client.16356 shutdown 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 msgr2=0x55d378146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f32a004b000 tx=0x7f32a002a9b0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 shutdown_connections 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d377d18da0 0x55d3781bf4f0 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d377d64390 0x55d378146f90 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f328eff5640 0 client.16356 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d377d85070 0x55d3781476a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 --2- 172.21.15.16:0/1155730363 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d377d5c1b0 0x55d378158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 >> 172.21.15.16:0/1155730363 conn(0x55d377d79460 msgr2=0x55d37813b0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.285+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 shutdown_connections 2024-09-21T03:08:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.286+0000 7f32b76b4740 1 -- 172.21.15.16:0/1155730363 wait complete. 2024-09-21T03:08:50.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.286+0000 7f32b76b4740 20 client.16356 trim_cache size 0 max 16384 2024-09-21T03:08:50.289 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:08:50.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 msgr2=0x7f32940b8d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 0x7f32940b8d10 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f32ac107b40 tx=0x7f32a0003720 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 msgr2=0x7f32ac19ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac19ecc0 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f329c00db00 tx=0x7f329c00e0c0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 shutdown_connections 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f32940b6800 0x7f32940b8d10 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f32ac108cf0 0x7f32ac19ecc0 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f32ac1083a0 0x7f32ac199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 --2- 172.21.15.16:0/1223426753 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f32ac102a60 0x7f32ac199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.287+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 >> 172.21.15.16:0/1223426753 conn(0x7f32ac079730 msgr2=0x7f32ac1052a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.288+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 shutdown_connections 2024-09-21T03:08:50.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:50.288+0000 7f32b1140640 1 -- 172.21.15.16:0/1223426753 wait complete. 2024-09-21T03:08:51.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:50 smithi016 ceph-mon[31253]: pgmap v578: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 69 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 469 MiB/s wr, 889 op/s 2024-09-21T03:08:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:50 smithi018 ceph-mon[34587]: pgmap v578: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 69 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 469 MiB/s wr, 889 op/s 2024-09-21T03:08:51.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:50 smithi123 ceph-mon[34645]: pgmap v578: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 69 GiB data, 209 GiB used, 863 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 469 MiB/s wr, 889 op/s 2024-09-21T03:08:52.137 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:08:52.138 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:08:52.621 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:53.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:52 smithi016 ceph-mon[31253]: pgmap v579: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 491 MiB/s wr, 962 op/s 2024-09-21T03:08:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:52 smithi018 ceph-mon[34587]: pgmap v579: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 491 MiB/s wr, 962 op/s 2024-09-21T03:08:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:52 smithi123 ceph-mon[34645]: pgmap v579: 97 pgs: 97 active+clean; 69 GiB data, 211 GiB used, 861 GiB / 1.0 TiB avail; 3.7 MiB/s rd, 491 MiB/s wr, 962 op/s 2024-09-21T03:08:53.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.723+0000 7f1ce8f19640 1 -- 172.21.15.16:0/3773971748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41079f0 msgr2=0x7f1ce4107ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.723+0000 7f1ce8f19640 1 --2- 172.21.15.16:0/3773971748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41079f0 0x7f1ce4107ed0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc009cd0 tx=0x7f1cdc02a930 comp rx=0 tx=0).stop 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 -- 172.21.15.16:0/3773971748 shutdown_connections 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 --2- 172.21.15.16:0/3773971748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41079f0 0x7f1ce4107ed0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 --2- 172.21.15.16:0/3773971748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce4102f00 0x7f1ce4107430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 --2- 172.21.15.16:0/3773971748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce41025b0 0x7f1ce41029c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 -- 172.21.15.16:0/3773971748 >> 172.21.15.16:0/3773971748 conn(0x7f1ce40fe2c0 msgr2=0x7f1ce4100710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 -- 172.21.15.16:0/3773971748 shutdown_connections 2024-09-21T03:08:53.729 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.726+0000 7f1ce8f19640 1 -- 172.21.15.16:0/3773971748 wait complete. 2024-09-21T03:08:53.730 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.728+0000 7f1ce8f19640 1 Processor -- start 2024-09-21T03:08:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 -- start start 2024-09-21T03:08:53.731 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41025b0 0x7f1ce41169c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 0x7f1ce410fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4119f40 con 0x7f1ce4102f00 2024-09-21T03:08:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4119dc0 con 0x7f1ce41025b0 2024-09-21T03:08:53.732 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.729+0000 7f1ce8f19640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce411a0c0 con 0x7f1ce41079f0 2024-09-21T03:08:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.731+0000 7f1ce3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.731+0000 7f1ce3fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44394/0 (socket says 172.21.15.16:44394) 2024-09-21T03:08:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.731+0000 7f1ce3fff640 1 -- 172.21.15.16:0/2343510468 learned_addr learned my addr 172.21.15.16:0/2343510468 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:53.733 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.731+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41025b0 0x7f1ce41169c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.736+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 0x7f1ce410fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce3fff640 1 -- 172.21.15.16:0/2343510468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41025b0 msgr2=0x7f1ce41169c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce3fff640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41025b0 0x7f1ce41169c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce3fff640 1 -- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 msgr2=0x7f1ce410fc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce3fff640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 0x7f1ce410fc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce3fff640 1 -- 172.21.15.16:0/2343510468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ce41108c0 con 0x7f1ce41079f0 2024-09-21T03:08:53.739 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.737+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 0x7f1ce410fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:08:53.749 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.745+0000 7f1ce3fff640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc006010 tx=0x7f1cdc02fc50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.747+0000 7f1ce0ff9640 1 -- 172.21.15.16:0/2343510468 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc0307f0 con 0x7f1ce41079f0 2024-09-21T03:08:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.748+0000 7f1ce0ff9640 1 -- 172.21.15.16:0/2343510468 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cdc02d070 con 0x7f1ce41079f0 2024-09-21T03:08:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.748+0000 7f1ce0ff9640 1 -- 172.21.15.16:0/2343510468 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cdc045c20 con 0x7f1ce41079f0 2024-09-21T03:08:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.748+0000 7f1ce8f19640 1 -- 172.21.15.16:0/2343510468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ce4115630 con 0x7f1ce41079f0 2024-09-21T03:08:53.754 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.749+0000 7f1ce8f19640 1 -- 172.21.15.16:0/2343510468 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ce4115b20 con 0x7f1ce41079f0 2024-09-21T03:08:53.761 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.756+0000 7f1ce0ff9640 1 -- 172.21.15.16:0/2343510468 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1cdc030990 con 0x7f1ce41079f0 2024-09-21T03:08:53.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.760+0000 7f1ce0ff9640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 0x7f1ccc0b8cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.761+0000 7f1ce0ff9640 1 -- 172.21.15.16:0/2343510468 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f1cdc109430 con 0x7f1ce41079f0 2024-09-21T03:08:53.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.762+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 0x7f1ccc0b8cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.766+0000 7f1cf06ef740 1 Processor -- start 2024-09-21T03:08:53.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.766+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 0x7f1ccc0b8cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400abe0 tx=0x7f1cd400a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.768+0000 7f1cf06ef740 1 -- start start 2024-09-21T03:08:53.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.769+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.770+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f7ffb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.771+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.771+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34808/0 (socket says 172.21.15.16:34808) 2024-09-21T03:08:53.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.772+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f800080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.772+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2376200552 learned_addr learned my addr 172.21.15.16:0/2376200552 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.772+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55584f775e60 con 0x55584f458350 2024-09-21T03:08:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.772+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55584f765290 con 0x55584f437670 2024-09-21T03:08:53.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.773+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584f7690a0 con 0x55584f42f490 2024-09-21T03:08:53.775 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.774+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2376200552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 msgr2=0x55584f7ffb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.774+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f7ffb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.774+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2376200552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 msgr2=0x55584f800080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.776 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.774+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f800080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.774+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2376200552 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584f4c3930 con 0x55584f458350 2024-09-21T03:08:53.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.775+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ffd0 tx=0x7f1cd4012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b983eb6c219d1cc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.777+0000 7f1cc9ffb640 1 -- 172.21.15.16:0/2376200552 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4016070 con 0x55584f458350 2024-09-21T03:08:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.778+0000 7f1cc9ffb640 1 -- 172.21.15.16:0/2376200552 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd401b8d0 con 0x55584f458350 2024-09-21T03:08:53.779 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.778+0000 7f1cc9ffb640 1 -- 172.21.15.16:0/2376200552 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd401c950 con 0x55584f458350 2024-09-21T03:08:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.784+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 msgr2=0x55584f7ff600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.784+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ffd0 tx=0x7f1cd4012000 comp rx=0 tx=0).stop 2024-09-21T03:08:53.789 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.785+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 shutdown_connections 2024-09-21T03:08:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.785+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f800080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.785+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f7ffb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.785+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2376200552 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f458350 0x55584f7ff600 secure :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ffd0 tx=0x7f1cd4012000 comp rx=0 tx=0).stop 2024-09-21T03:08:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.785+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 >> 172.21.15.16:0/2376200552 conn(0x55584f44c740 msgr2=0x55584f7fd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.790 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.789+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 shutdown_connections 2024-09-21T03:08:53.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.791+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2376200552 wait complete. 2024-09-21T03:08:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.792+0000 7f1cf06ef740 1 -- start start 2024-09-21T03:08:53.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.793+0000 7f1cf06ef740 1 Processor -- start 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.794+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.794+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f437670 0x55584f80b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.794+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f8110a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.794+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55584f5c6890 con 0x55584f437670 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.794+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55584f6ffd40 con 0x55584f416cd0 2024-09-21T03:08:53.796 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.795+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55584f80dde0 con 0x55584f42f490 2024-09-21T03:08:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.795+0000 7f1cf06ef740 10 client.0 _ll_register_callbacks cb 0x55584f763c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.795+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.795+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:44410/0 (socket says 172.21.15.16:44410) 2024-09-21T03:08:53.797 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.795+0000 7f1ce37fe640 1 -- 172.21.15.16:0/821020070 learned_addr learned my addr 172.21.15.16:0/821020070 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.797+0000 7f1ce37fe640 1 -- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 msgr2=0x55584f8110a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.797+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f8110a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.797+0000 7f1ce37fe640 1 -- 172.21.15.16:0/821020070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f437670 msgr2=0x55584f80b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.797+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f437670 0x55584f80b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.797+0000 7f1ce37fe640 1 -- 172.21.15.16:0/821020070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584f811710 con 0x55584f42f490 2024-09-21T03:08:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.798+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ffa0 tx=0x7f1cd401b7b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b32cc19f320719f0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.800 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.799+0000 7f1cc2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.800+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4033060 con 0x55584f42f490 2024-09-21T03:08:53.802 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.801+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1cd401b8d0 con 0x55584f42f490 2024-09-21T03:08:53.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.801+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f1cd401bd80 con 0x55584f42f490 2024-09-21T03:08:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.823+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1cd402a040 con 0x55584f42f490 2024-09-21T03:08:53.824 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.823+0000 7f1cf06ef740 10 client.25993 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:53.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.823+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55584f81a1b0 con 0x55584f42f490 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.825+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd401b8d0 con 0x55584f42f490 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1cd402d030 con 0x55584f42f490 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 10 client.25993 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 10 client.25993 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 20 client.25993 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:53.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f3ec080 0x55584f81d920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 4 client.25993 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:53.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.826+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55584f777db0 con 0x55584f3ec080 2024-09-21T03:08:53.835 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.833+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f3ec080 0x55584f81d920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.839+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f3ec080 0x55584f81d920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27cd2065f8e03b9c server_cookie=f75d56edc47abaab in_seq=0 out_seq=0 2024-09-21T03:08:53.841 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.839+0000 7f1cc2ffd640 10 client.25993 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.841+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/821020070 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55584f777db0 con 0x55584f3ec080 2024-09-21T03:08:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.841+0000 7f1cc2ffd640 10 client.25993 handle_command_reply: tid=0 2024-09-21T03:08:53.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.841+0000 7f1cf06ef740 1 client.25993 shutdown 2024-09-21T03:08:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.841+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 msgr2=0x55584f80b3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.841+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400ffa0 tx=0x7f1cd401b7b0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 shutdown_connections 2024-09-21T03:08:53.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f3ec080 0x55584f81d920 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f437670 0x55584f80b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f8110a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cc2ffd640 0 client.25993 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/821020070 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f80b3b0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 >> 172.21.15.16:0/821020070 conn(0x55584f44c740 msgr2=0x55584f800630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 shutdown_connections 2024-09-21T03:08:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.842+0000 7f1cf06ef740 1 -- 172.21.15.16:0/821020070 wait complete. 2024-09-21T03:08:53.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.843+0000 7f1cf06ef740 20 client.25993 trim_cache size 0 max 16384 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.871+0000 7f1cf06ef740 1 Processor -- start 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.871+0000 7f1cf06ef740 1 -- start start 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.871+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.871+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f8083c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.871+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f808900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4072dd0 con 0x55584f416cd0 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce41c74b0 con 0x55584f437670 2024-09-21T03:08:53.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1cf06ef740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4073420 con 0x55584f42f490 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34824/0 (socket says 172.21.15.16:34824) 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2168374562 learned_addr learned my addr 172.21.15.16:0/2168374562 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2168374562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 msgr2=0x55584f8083c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f8083c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2168374562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 msgr2=0x55584f808900 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f808900 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.872+0000 7f1ce37fe640 1 -- 172.21.15.16:0/2168374562 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55584f80a110 con 0x55584f416cd0 2024-09-21T03:08:53.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.873+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4010d10 tx=0x7f1cd400a380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ba888df1cc40842 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.873+0000 7f1cc97fa640 1 -- 172.21.15.16:0/2168374562 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4016040 con 0x55584f416cd0 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.873+0000 7f1cc97fa640 1 -- 172.21.15.16:0/2168374562 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1cd400a830 con 0x55584f416cd0 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.873+0000 7f1cc97fa640 1 -- 172.21.15.16:0/2168374562 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd4033040 con 0x55584f416cd0 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.874+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2168374562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 msgr2=0x55584f800d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.874+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f1cd4010d10 tx=0x7f1cd400a380 comp rx=0 tx=0).stop 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2168374562 shutdown_connections 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f437670 0x55584f808900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f42f490 0x55584f8083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/2168374562 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f416cd0 0x55584f800d40 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2168374562 >> 172.21.15.16:0/2168374562 conn(0x55584f44c740 msgr2=0x55584f800630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2168374562 shutdown_connections 2024-09-21T03:08:53.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.875+0000 7f1cf06ef740 1 -- 172.21.15.16:0/2168374562 wait complete. 2024-09-21T03:08:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.876+0000 7f1cf06ef740 1 -- start start 2024-09-21T03:08:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.876+0000 7f1cf06ef740 1 Processor -- start 2024-09-21T03:08:53.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.877+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f817fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.877+0000 7f1cf06ef740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f817fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce2ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:34836/0 (socket says 172.21.15.16:34836) 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce37fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f817fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:43828/0 (socket says 172.21.15.16:43828) 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce2ffd640 1 -- 172.21.15.16:0/12944985 learned_addr learned my addr 172.21.15.16:0/12944985 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f437670 0x55584f815c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4073af0 con 0x55584f42f490 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce4074f60 con 0x55584f416cd0 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.878+0000 7f1ce3fff640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f437670 0x55584f815c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1ce40741c0 con 0x55584f437670 2024-09-21T03:08:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1cf06ef740 10 client.0 _ll_register_callbacks cb 0x7f1cd400fea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1ce2ffd640 1 -- 172.21.15.16:0/12944985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f437670 msgr2=0x55584f815c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f437670 0x55584f815c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1ce2ffd640 1 -- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 msgr2=0x55584f817fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:53.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f817fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.879+0000 7f1ce2ffd640 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55584f887330 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55584f773860 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1ce2ffd640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800a750 tx=0x7f1cd800ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35640f41a2bba77f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cc2ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/12944985 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1cd801e020 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/12944985 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f1cd8013ce0 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/12944985 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1cd801a440 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cf06ef740 10 client.16365 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.880+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({fsmap=20}) -- 0x55584f888450 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/12944985 <== mon.0 v2:172.21.15.16:3300/0 4 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1cd801a5e0 con 0x55584f42f490 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 10 client.16365 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 10 client.16365 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:53.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 20 client.16365 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f888b10 0x55584f88af30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 4 client.16365 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55584f770c90 con 0x55584f888b10 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.881+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f888b10 0x55584f88af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.882+0000 7f1cc2ffd640 10 client.16365 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:53.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.882+0000 7f1ce37fe640 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f888b10 0x55584f88af30 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a1f2a7dbfc287ef server_cookie=1783c519b9f3a14a in_seq=0 out_seq=0 2024-09-21T03:08:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cc2ffd640 1 -- 172.21.15.16:0/12944985 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55584f770c90 con 0x55584f888b10 2024-09-21T03:08:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cc2ffd640 10 client.16365 handle_command_reply: tid=0 2024-09-21T03:08:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 client.16365 shutdown 2024-09-21T03:08:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 msgr2=0x55584f815510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.886 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f1cd800a750 tx=0x7f1cd800ad80 comp rx=0 tx=0).stop 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 shutdown_connections 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55584f888b10 0x55584f88af30 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55584f42f490 0x55584f815510 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55584f437670 0x55584f815c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cc2ffd640 0 client.16365 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.884+0000 7f1cf06ef740 1 --2- 172.21.15.16:0/12944985 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55584f416cd0 0x55584f817fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.885+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 >> 172.21.15.16:0/12944985 conn(0x55584f44c740 msgr2=0x55584f80c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.885+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 shutdown_connections 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.885+0000 7f1cf06ef740 1 -- 172.21.15.16:0/12944985 wait complete. 2024-09-21T03:08:53.887 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.885+0000 7f1cf06ef740 20 client.16365 trim_cache size 0 max 16384 2024-09-21T03:08:53.888 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:08:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "6acce003-5dce-479a-a188-c4c69ecf4f2f", 2024-09-21T03:08:53.888 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:08:53.888 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 msgr2=0x7f1ccc0b8cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 0x7f1ccc0b8cc0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f1cd400abe0 tx=0x7f1cd400a2d0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 msgr2=0x7f1ce41101d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:53.889 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f1cdc006010 tx=0x7f1cdc02fc50 comp rx=0 tx=0).stop 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 shutdown_connections 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1ccc0b67b0 0x7f1ccc0b8cc0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1ce4102f00 0x7f1ce410fc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ce41079f0 0x7f1ce41101d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 --2- 172.21.15.16:0/2343510468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1ce41025b0 0x7f1ce41169c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 >> 172.21.15.16:0/2343510468 conn(0x7f1ce40fe2c0 msgr2=0x7f1ce4105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.886+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 shutdown_connections 2024-09-21T03:08:53.890 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:53.887+0000 7f1cca7fc640 1 -- 172.21.15.16:0/2343510468 wait complete. 2024-09-21T03:08:54.768 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:08:55.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:54 smithi016 ceph-mon[31253]: pgmap v580: 97 pgs: 97 active+clean; 70 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 444 MiB/s wr, 931 op/s 2024-09-21T03:08:55.069 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:08:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:54 smithi018 ceph-mon[34587]: pgmap v580: 97 pgs: 97 active+clean; 70 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 444 MiB/s wr, 931 op/s 2024-09-21T03:08:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:54 smithi123 ceph-mon[34645]: pgmap v580: 97 pgs: 97 active+clean; 70 GiB data, 212 GiB used, 861 GiB / 1.0 TiB avail; 2.5 MiB/s rd, 444 MiB/s wr, 931 op/s 2024-09-21T03:08:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.760+0000 7f3617cd2640 1 -- 172.21.15.16:0/2072362313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610111f40 msgr2=0x7f361010a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:55.764 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.760+0000 7f3617cd2640 1 --2- 172.21.15.16:0/2072362313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610111f40 0x7f361010a3c0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f360001a910 tx=0x7f360003b450 comp rx=0 tx=0).stop 2024-09-21T03:08:55.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.764+0000 7f3617cd2640 1 -- 172.21.15.16:0/2072362313 shutdown_connections 2024-09-21T03:08:55.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.764+0000 7f3617cd2640 1 --2- 172.21.15.16:0/2072362313 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361011cc30 0x7f361011f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.764+0000 7f3617cd2640 1 --2- 172.21.15.16:0/2072362313 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f361010a9e0 0x7f361010adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.764+0000 7f3617cd2640 1 --2- 172.21.15.16:0/2072362313 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3610111f40 0x7f361010a3c0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.764+0000 7f3617cd2640 1 -- 172.21.15.16:0/2072362313 >> 172.21.15.16:0/2072362313 conn(0x7f3610070ec0 msgr2=0x7f36100712d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:55.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.769+0000 7f3617cd2640 1 -- 172.21.15.16:0/2072362313 shutdown_connections 2024-09-21T03:08:55.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.769+0000 7f3617cd2640 1 -- 172.21.15.16:0/2072362313 wait complete. 2024-09-21T03:08:55.772 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 Processor -- start 2024-09-21T03:08:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 -- start start 2024-09-21T03:08:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3610111f40 0x7f3610118650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.773 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361011cc30 0x7f3610118b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f3610114c10 con 0x7f361011cc30 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f3610114a90 con 0x7f3610111f40 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.771+0000 7f3617cd2640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f3610114d90 con 0x7f361010a9e0 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.772+0000 7f3615a6e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.772+0000 7f3615a6e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42542/0 (socket says 172.21.15.16:42542) 2024-09-21T03:08:55.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.772+0000 7f3615a6e640 1 -- 172.21.15.16:0/4199107537 learned_addr learned my addr 172.21.15.16:0/4199107537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.775+0000 7f3615a6e640 1 -- 172.21.15.16:0/4199107537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3610111f40 msgr2=0x7f3610118650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.775+0000 7f3615a6e640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3610111f40 0x7f3610118650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.775+0000 7f3615a6e640 1 -- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361011cc30 msgr2=0x7f3610118b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.775+0000 7f3615a6e640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361011cc30 0x7f3610118b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.775+0000 7f3615a6e640 1 -- 172.21.15.16:0/4199107537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36101192a0 con 0x7f361010a9e0 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.776+0000 7f3615a6e640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f360001a860 tx=0x7f36000048d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.777+0000 7f3606ffd640 1 -- 172.21.15.16:0/4199107537 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3600052020 con 0x7f361010a9e0 2024-09-21T03:08:55.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.777+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36101acdd0 con 0x7f361010a9e0 2024-09-21T03:08:55.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.777+0000 7f3606ffd640 1 -- 172.21.15.16:0/4199107537 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3600040d10 con 0x7f361010a9e0 2024-09-21T03:08:55.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.777+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36101ad1d0 con 0x7f361010a9e0 2024-09-21T03:08:55.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.783+0000 7f361e246740 1 Processor -- start 2024-09-21T03:08:55.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.783+0000 7f3606ffd640 1 -- 172.21.15.16:0/4199107537 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360004ee20 con 0x7f361010a9e0 2024-09-21T03:08:55.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.784+0000 7f361e246740 1 -- start start 2024-09-21T03:08:55.792 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.785+0000 7f361e246740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.787+0000 7f3606ffd640 1 -- 172.21.15.16:0/4199107537 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f3600055020 con 0x7f361010a9e0 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.789+0000 7f3606ffd640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 0x7f35fc0b8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.790+0000 7f3606ffd640 1 -- 172.21.15.16:0/4199107537 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f3600119410 con 0x7f361010a9e0 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361e246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x5557741277b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361e246740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x555774127cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361e246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55577409de50 con 0x555773d571b0 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361e246740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55577408d280 con 0x555773d80070 2024-09-21T03:08:55.793 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361e246740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x555774091090 con 0x555773d5f390 2024-09-21T03:08:55.794 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.791+0000 7f361626f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.798 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.793+0000 7f361626f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:55540/0 (socket says 172.21.15.16:55540) 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.793+0000 7f361626f640 1 -- 172.21.15.16:0/1493213932 learned_addr learned my addr 172.21.15.16:0/1493213932 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.793+0000 7f361626f640 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x555774127cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f361626f640 1 -- 172.21.15.16:0/1493213932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 msgr2=0x555774127cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f361626f640 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x555774127cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f361626f640 1 -- 172.21.15.16:0/1493213932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 msgr2=0x5557741277b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f361626f640 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x5557741277b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f361626f640 1 -- 172.21.15.16:0/1493213932 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555773deb650 con 0x555773d80070 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.795+0000 7f3615a6e640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 0x7f35fc0b8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.799 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.796+0000 7f361626f640 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3608005c40 tx=0x7f36080099b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=acb56b44c257cda5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:55.801 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.798+0000 7f3615a6e640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 0x7f35fc0b8bb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3600116b20 tx=0x7f36000c4f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:55.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.800+0000 7f35fbfff640 1 -- 172.21.15.16:0/1493213932 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608013230 con 0x555773d80070 2024-09-21T03:08:55.803 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.800+0000 7f35fbfff640 1 -- 172.21.15.16:0/1493213932 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360800c850 con 0x555773d80070 2024-09-21T03:08:55.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.800+0000 7f35fbfff640 1 -- 172.21.15.16:0/1493213932 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3608005310 con 0x555773d80070 2024-09-21T03:08:55.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.802+0000 7f361e246740 1 -- 172.21.15.16:0/1493213932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 msgr2=0x555774127270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:55.804 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.802+0000 7f361e246740 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f3608005c40 tx=0x7f36080099b0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.810+0000 7f361e246740 1 -- 172.21.15.16:0/1493213932 shutdown_connections 2024-09-21T03:08:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.810+0000 7f361e246740 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x5557741277b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.810+0000 7f361e246740 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x555774127cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.810+0000 7f361e246740 1 --2- 172.21.15.16:0/1493213932 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774127270 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.813 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.810+0000 7f361e246740 1 -- 172.21.15.16:0/1493213932 >> 172.21.15.16:0/1493213932 conn(0x555773d74460 msgr2=0x555774125160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.814+0000 7f361e246740 1 -- 172.21.15.16:0/1493213932 shutdown_connections 2024-09-21T03:08:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.814+0000 7f361e246740 1 -- 172.21.15.16:0/1493213932 wait complete. 2024-09-21T03:08:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.815+0000 7f361e246740 1 -- start start 2024-09-21T03:08:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.816+0000 7f361e246740 1 Processor -- start 2024-09-21T03:08:55.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 0x555774132910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 0x555774132e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x5557741333d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x555773eee700 con 0x555773d5f390 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x555774028110 con 0x555773d80070 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5557741359f0 con 0x555773d571b0 2024-09-21T03:08:55.825 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.817+0000 7f361e246740 10 client.0 _ll_register_callbacks cb 0x555774041990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.819+0000 7f361626f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 0x555774132910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.819+0000 7f361626f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 0x555774132910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42554/0 (socket says 172.21.15.16:42554) 2024-09-21T03:08:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.819+0000 7f361626f640 1 -- 172.21.15.16:0/2946569289 learned_addr learned my addr 172.21.15.16:0/2946569289 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.819+0000 7f361626f640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x5557741333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.821+0000 7f361526d640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 0x555774132e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.824+0000 7f361526d640 1 -- 172.21.15.16:0/2946569289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 msgr2=0x555774132910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.824+0000 7f361526d640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 0x555774132910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.824+0000 7f361526d640 1 -- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 msgr2=0x5557741333d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.824+0000 7f361526d640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x5557741333d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.824+0000 7f361526d640 1 -- 172.21.15.16:0/2946569289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555774055c40 con 0x555773d5f390 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.825+0000 7f361526d640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 0x555774132e50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f360c005940 tx=0x7f360c00af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a15016ee9cfec25 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:55.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.825+0000 7f35f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:08:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.826+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00f860 con 0x555773d5f390 2024-09-21T03:08:55.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.826+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360800c850 con 0x555773d5f390 2024-09-21T03:08:55.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.827+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f360c00fa00 con 0x555773d5f390 2024-09-21T03:08:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.829+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f360c00fc80 con 0x555773d5f390 2024-09-21T03:08:55.831 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.829+0000 7f361e246740 10 client.16371 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:55.832 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.829+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x555774141800 con 0x555773d5f390 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.833+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360800c850 con 0x555773d5f390 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.835+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f360c00e3d0 con 0x555773d5f390 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.835+0000 7f361e246740 10 client.16371 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.835+0000 7f361e246740 10 client.16371 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.835+0000 7f361e246740 20 client.16371 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.835+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d3e9f0 0x555774144ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.837+0000 7f361e246740 4 client.16371 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:08:55.839 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.837+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55577409fda0 con 0x555773d3e9f0 2024-09-21T03:08:55.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.850+0000 7f361626f640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d3e9f0 0x555774144ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.860+0000 7f361626f640 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d3e9f0 0x555774144ce0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4df6c8bbd241d41 server_cookie=ded5f368656a4ac5 in_seq=0 out_seq=0 2024-09-21T03:08:55.863 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.860+0000 7f35f97fa640 10 client.16371 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.864+0000 7f35f97fa640 1 -- 172.21.15.16:0/2946569289 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55577409fda0 con 0x555773d3e9f0 2024-09-21T03:08:55.866 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.864+0000 7f35f97fa640 10 client.16371 handle_command_reply: tid=0 2024-09-21T03:08:55.867 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.864+0000 7f361e246740 1 client.16371 shutdown 2024-09-21T03:08:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 msgr2=0x555774132e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 0x555774132e50 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f360c005940 tx=0x7f360c00af90 comp rx=0 tx=0).stop 2024-09-21T03:08:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 shutdown_connections 2024-09-21T03:08:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d3e9f0 0x555774144ce0 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.869 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d5f390 0x555774132e50 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x5557741333d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 --2- 172.21.15.16:0/2946569289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d571b0 0x555774132910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.866+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 >> 172.21.15.16:0/2946569289 conn(0x555773d74460 msgr2=0x55577412a400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:55.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.867+0000 7f35f97fa640 0 client.16371 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:55.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.867+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 shutdown_connections 2024-09-21T03:08:55.871 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.867+0000 7f361e246740 1 -- 172.21.15.16:0/2946569289 wait complete. 2024-09-21T03:08:55.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.871+0000 7f361e246740 20 client.16371 trim_cache size 0 max 16384 2024-09-21T03:08:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.915+0000 7f361e246740 1 Processor -- start 2024-09-21T03:08:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.915+0000 7f361e246740 1 -- start start 2024-09-21T03:08:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.916+0000 7f361e246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x555774132dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.919 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.917+0000 7f361e246740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.920 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.917+0000 7f361e246740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774130040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.917+0000 7f361e246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400c820 con 0x555773d571b0 2024-09-21T03:08:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.917+0000 7f361526d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.918+0000 7f361526d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42568/0 (socket says 172.21.15.16:42568) 2024-09-21T03:08:55.923 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.918+0000 7f361526d640 1 -- 172.21.15.16:0/3689417738 learned_addr learned my addr 172.21.15.16:0/3689417738 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.918+0000 7f361626f640 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774130040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.918+0000 7f361626f640 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x555774132dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.921+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400eff0 con 0x555773d80070 2024-09-21T03:08:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.921+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400cde0 con 0x555773d5f390 2024-09-21T03:08:55.924 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.922+0000 7f361526d640 1 -- 172.21.15.16:0/3689417738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 msgr2=0x555774130040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.922+0000 7f361526d640 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774130040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.922+0000 7f361526d640 1 -- 172.21.15.16:0/3689417738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 msgr2=0x555774132dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.922+0000 7f361526d640 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x555774132dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:55.925 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.922+0000 7f361526d640 1 -- 172.21.15.16:0/3689417738 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577412a0d0 con 0x555773d5f390 2024-09-21T03:08:55.931 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:55.924+0000 7f361526d640 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f360c012040 tx=0x7f360c00ccf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b16fdaed075f52b4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.107+0000 7f35fb7fe640 1 -- 172.21.15.16:0/3689417738 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00a2d0 con 0x555773d5f390 2024-09-21T03:08:56.108 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.107+0000 7f35fb7fe640 1 -- 172.21.15.16:0/3689417738 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c00edb0 con 0x555773d5f390 2024-09-21T03:08:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.107+0000 7f35fb7fe640 1 -- 172.21.15.16:0/3689417738 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c022420 con 0x555773d5f390 2024-09-21T03:08:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 msgr2=0x55577412fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f360c012040 tx=0x7f360c00ccf0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 shutdown_connections 2024-09-21T03:08:56.109 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774130040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577412fb00 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 --2- 172.21.15.16:0/3689417738 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x555774132dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 >> 172.21.15.16:0/3689417738 conn(0x555773d74460 msgr2=0x5557741326c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 shutdown_connections 2024-09-21T03:08:56.110 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- 172.21.15.16:0/3689417738 wait complete. 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.108+0000 7f361e246740 1 -- start start 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 Processor -- start 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577413b710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774136820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400d4b0 con 0x555773d571b0 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400e920 con 0x555773d80070 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f35f400db80 con 0x555773d5f390 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361e246740 10 client.0 _ll_register_callbacks cb 0x7f360c005e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:08:56.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361626f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361626f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:43638/0 (socket says 172.21.15.16:43638) 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.109+0000 7f361626f640 1 -- 172.21.15.16:0/2304839937 learned_addr learned my addr 172.21.15.16:0/2304839937 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577413b710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361626f640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774136820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 -- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 msgr2=0x555774136820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774136820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 -- 172.21.15.16:0/2304839937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 msgr2=0x55577413b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:08:56.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 -- 172.21.15.16:0/2304839937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555774136fb0 con 0x555773d5f390 2024-09-21T03:08:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361626f640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:08:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f361526d640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577413b710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f360c00e770 tx=0x7f360c00f910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84543eaedc6fbd38 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:08:56.113 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.110+0000 7f35f97fa640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:08:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f360c00e3b0 con 0x555773d5f390 2024-09-21T03:08:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(82..82 src has 1..82) ==== 8580+0+0 (secure 0 0 0) 0x7f360c004020 con 0x555773d5f390 2024-09-21T03:08:56.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f360c00edb0 con 0x555773d5f390 2024-09-21T03:08:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f360c00a300 con 0x555773d5f390 2024-09-21T03:08:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f361e246740 10 client.26014 fetch_fsmap learned FSMap version 20 2024-09-21T03:08:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.113+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5557741af2d0 con 0x555773d5f390 2024-09-21T03:08:56.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.114+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f360c00edb0 con 0x555773d5f390 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.114+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f360c022ae0 con 0x555773d5f390 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.114+0000 7f361e246740 10 client.26014 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.114+0000 7f361e246740 10 client.26014 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.114+0000 7f361e246740 20 client.26014 populate_metadata read hostname 'smithi016' 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d13da0 0x5557741b1970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361e246740 4 client.26014 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55577409b850 con 0x555773d13da0 2024-09-21T03:08:56.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361626f640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d13da0 0x5557741b1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:08:56.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f35f97fa640 10 client.26014 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361626f640 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d13da0 0x5557741b1970 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a94d9717a4c16e25 server_cookie=fe569ac5d63fe642 in_seq=0 out_seq=0 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f35f97fa640 1 -- 172.21.15.16:0/2304839937 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4830 (crc 0 0 0) 0x55577409b850 con 0x555773d13da0 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f35f97fa640 10 client.26014 handle_command_reply: tid=0 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.115+0000 7f361e246740 1 client.26014 shutdown 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 msgr2=0x55577413b710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577413b710 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f360c00e770 tx=0x7f360c00f910 comp rx=0 tx=0).stop 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 shutdown_connections 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x555773d13da0 0x5557741b1970 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x555773d80070 0x555774136820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x555773d5f390 0x55577413b710 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f35f97fa640 0 client.26014 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 --2- 172.21.15.16:0/2304839937 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x555773d571b0 0x55577413b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.118 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 >> 172.21.15.16:0/2304839937 conn(0x555773d74460 msgr2=0x5557741282a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 shutdown_connections 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 1 -- 172.21.15.16:0/2304839937 wait complete. 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.116+0000 7f361e246740 20 client.26014 trim_cache size 0 max 16384 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:08:56.119 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "6acce003-5dce-479a-a188-c4c69ecf4f2f", 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:08:56.120 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:08:56.122 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:08:56.124 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.125 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.126 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.128 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:08:56.129 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:08:56.130 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 msgr2=0x7f35fc0b8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 0x7f35fc0b8bb0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f3600116b20 tx=0x7f36000c4f90 comp rx=0 tx=0).stop 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 msgr2=0x7f361011c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f360001a860 tx=0x7f36000048d0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 shutdown_connections 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f35fc0b66a0 0x7f35fc0b8bb0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f361011cc30 0x7f3610118b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f3610111f40 0x7f3610118650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 --2- 172.21.15.16:0/4199107537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f361010a9e0 0x7f361011c370 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:08:56.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 >> 172.21.15.16:0/4199107537 conn(0x7f3610070ec0 msgr2=0x7f361010ba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:08:56.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 shutdown_connections 2024-09-21T03:08:56.132 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:08:56.117+0000 7f3617cd2640 1 -- 172.21.15.16:0/4199107537 wait complete. 2024-09-21T03:08:56.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:56 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:08:56.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:56 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:56 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:08:56.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:56 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:56 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:08:56.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:56 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.453 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:56 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:08:56.454 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:56 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:56 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:08:56.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:56 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:56 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:08:56.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:56 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:08:56.971 INFO:tasks.cephfs.filesystem:scrub status for tag:6acce003-5dce-479a-a188-c4c69ecf4f2f - {'path': '/', 'tag': '6acce003-5dce-479a-a188-c4c69ecf4f2f', 'options': 'recursive,force'} 2024-09-21T03:08:57.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:57 smithi016 ceph-mon[31253]: pgmap v581: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 530 MiB/s wr, 1.03k op/s 2024-09-21T03:08:57.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:57 smithi016 ceph-mon[31253]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:08:57.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:57 smithi018 ceph-mon[34587]: pgmap v581: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 530 MiB/s wr, 1.03k op/s 2024-09-21T03:08:57.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:57 smithi018 ceph-mon[34587]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:08:57.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:57 smithi123 ceph-mon[34645]: pgmap v581: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.3 MiB/s rd, 530 MiB/s wr, 1.03k op/s 2024-09-21T03:08:57.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:57 smithi123 ceph-mon[34645]: osdmap e83: 12 total, 12 up, 12 in 2024-09-21T03:08:58.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:58 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:08:58.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:58 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:08:58.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:58 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:08:59.416 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:08:59 smithi016 ceph-mon[31253]: pgmap v583: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 507 MiB/s wr, 831 op/s 2024-09-21T03:08:59.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:08:59 smithi018 ceph-mon[34587]: pgmap v583: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 507 MiB/s wr, 831 op/s 2024-09-21T03:08:59.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:08:59 smithi123 ceph-mon[34645]: pgmap v583: 97 pgs: 97 active+clean; 70 GiB data, 214 GiB used, 859 GiB / 1.0 TiB avail; 3.8 MiB/s rd, 507 MiB/s wr, 831 op/s 2024-09-21T03:09:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:01 smithi018 ceph-mon[34587]: pgmap v584: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 509 MiB/s wr, 793 op/s 2024-09-21T03:09:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:01 smithi123 ceph-mon[34645]: pgmap v584: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 509 MiB/s wr, 793 op/s 2024-09-21T03:09:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:01 smithi016 ceph-mon[31253]: pgmap v584: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.5 MiB/s rd, 509 MiB/s wr, 793 op/s 2024-09-21T03:09:03.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:03 smithi018 ceph-mon[34587]: pgmap v585: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 476 MiB/s wr, 737 op/s 2024-09-21T03:09:03.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:03 smithi123 ceph-mon[34645]: pgmap v585: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 476 MiB/s wr, 737 op/s 2024-09-21T03:09:03.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:03 smithi016 ceph-mon[31253]: pgmap v585: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.0 MiB/s rd, 476 MiB/s wr, 737 op/s 2024-09-21T03:09:05.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:05 smithi018 ceph-mon[34587]: pgmap v586: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 448 MiB/s wr, 716 op/s 2024-09-21T03:09:05.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:05 smithi123 ceph-mon[34645]: pgmap v586: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 448 MiB/s wr, 716 op/s 2024-09-21T03:09:05.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:05 smithi016 ceph-mon[31253]: pgmap v586: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 3.6 MiB/s rd, 448 MiB/s wr, 716 op/s 2024-09-21T03:09:07.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:07 smithi018 ceph-mon[34587]: pgmap v587: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 388 MiB/s wr, 704 op/s 2024-09-21T03:09:07.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:07 smithi123 ceph-mon[34645]: pgmap v587: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 388 MiB/s wr, 704 op/s 2024-09-21T03:09:07.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:07 smithi016 ceph-mon[31253]: pgmap v587: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.6 MiB/s rd, 388 MiB/s wr, 704 op/s 2024-09-21T03:09:09.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:09 smithi123 ceph-mon[34645]: pgmap v588: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 339 MiB/s wr, 657 op/s 2024-09-21T03:09:09.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:09 smithi123 ceph-mon[34645]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:09:09.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:09 smithi016 ceph-mon[31253]: pgmap v588: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 339 MiB/s wr, 657 op/s 2024-09-21T03:09:09.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:09 smithi016 ceph-mon[31253]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:09:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:09 smithi018 ceph-mon[34587]: pgmap v588: 97 pgs: 97 active+clean; 70 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 2.0 MiB/s rd, 339 MiB/s wr, 657 op/s 2024-09-21T03:09:09.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:09 smithi018 ceph-mon[34587]: osdmap e84: 12 total, 12 up, 12 in 2024-09-21T03:09:11.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:11 smithi123 ceph-mon[34645]: pgmap v590: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 69 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 306 MiB/s wr, 657 op/s 2024-09-21T03:09:11.484 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:11 smithi018 ceph-mon[34587]: pgmap v590: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 69 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 306 MiB/s wr, 657 op/s 2024-09-21T03:09:11.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:11 smithi016 ceph-mon[31253]: pgmap v590: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 69 GiB data, 215 GiB used, 858 GiB / 1.0 TiB avail; 1.7 MiB/s rd, 306 MiB/s wr, 657 op/s 2024-09-21T03:09:13.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:13 smithi016 ceph-mon[31253]: pgmap v591: 97 pgs: 19 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 49 active+clean; 68 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 756 KiB/s rd, 233 MiB/s wr, 572 op/s 2024-09-21T03:09:13.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:13 smithi018 ceph-mon[34587]: pgmap v591: 97 pgs: 19 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 49 active+clean; 68 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 756 KiB/s rd, 233 MiB/s wr, 572 op/s 2024-09-21T03:09:13.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:13 smithi123 ceph-mon[34645]: pgmap v591: 97 pgs: 19 active+clean+snaptrim, 29 active+clean+snaptrim_wait, 49 active+clean; 68 GiB data, 211 GiB used, 862 GiB / 1.0 TiB avail; 756 KiB/s rd, 233 MiB/s wr, 572 op/s 2024-09-21T03:09:14.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:14 smithi018 ceph-mon[34587]: pgmap v592: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 68 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 115 KiB/s rd, 200 MiB/s wr, 531 op/s 2024-09-21T03:09:14.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:14 smithi123 ceph-mon[34645]: pgmap v592: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 68 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 115 KiB/s rd, 200 MiB/s wr, 531 op/s 2024-09-21T03:09:14.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:14 smithi016 ceph-mon[31253]: pgmap v592: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 68 GiB data, 210 GiB used, 863 GiB / 1.0 TiB avail; 115 KiB/s rd, 200 MiB/s wr, 531 op/s 2024-09-21T03:09:16.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:16 smithi018 ceph-mon[34587]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 114 KiB/s rd, 135 MiB/s wr, 491 op/s 2024-09-21T03:09:16.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:16 smithi123 ceph-mon[34645]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 114 KiB/s rd, 135 MiB/s wr, 491 op/s 2024-09-21T03:09:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:16 smithi016 ceph-mon[31253]: pgmap v593: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 114 KiB/s rd, 135 MiB/s wr, 491 op/s 2024-09-21T03:09:18.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:18 smithi018 ceph-mon[34587]: pgmap v594: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 205 GiB used, 867 GiB / 1.0 TiB avail; 48 KiB/s rd, 166 MiB/s wr, 809 op/s 2024-09-21T03:09:18.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:18 smithi123 ceph-mon[34645]: pgmap v594: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 205 GiB used, 867 GiB / 1.0 TiB avail; 48 KiB/s rd, 166 MiB/s wr, 809 op/s 2024-09-21T03:09:19.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:18 smithi016 ceph-mon[31253]: pgmap v594: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 205 GiB used, 867 GiB / 1.0 TiB avail; 48 KiB/s rd, 166 MiB/s wr, 809 op/s 2024-09-21T03:09:20.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:20 smithi018 ceph-mon[34587]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 42 KiB/s rd, 230 MiB/s wr, 1.34k op/s 2024-09-21T03:09:20.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:20 smithi123 ceph-mon[34645]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 42 KiB/s rd, 230 MiB/s wr, 1.34k op/s 2024-09-21T03:09:21.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:20 smithi016 ceph-mon[31253]: pgmap v595: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 67 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 42 KiB/s rd, 230 MiB/s wr, 1.34k op/s 2024-09-21T03:09:21.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:21 smithi123 ceph-mon[34645]: Health check update: 4 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:09:22.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:21 smithi016 ceph-mon[31253]: Health check update: 4 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:09:22.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:21 smithi018 ceph-mon[34587]: Health check update: 4 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:09:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:22 smithi123 ceph-mon[34645]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 45 KiB/s rd, 296 MiB/s wr, 1.84k op/s 2024-09-21T03:09:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:22 smithi016 ceph-mon[31253]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 45 KiB/s rd, 296 MiB/s wr, 1.84k op/s 2024-09-21T03:09:23.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:22 smithi018 ceph-mon[34587]: pgmap v596: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 206 GiB used, 867 GiB / 1.0 TiB avail; 45 KiB/s rd, 296 MiB/s wr, 1.84k op/s 2024-09-21T03:09:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:24 smithi123 ceph-mon[34645]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 275 MiB/s wr, 1.79k op/s 2024-09-21T03:09:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:24 smithi016 ceph-mon[31253]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 275 MiB/s wr, 1.79k op/s 2024-09-21T03:09:25.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:24 smithi018 ceph-mon[34587]: pgmap v597: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 66 GiB data, 205 GiB used, 868 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 275 MiB/s wr, 1.79k op/s 2024-09-21T03:09:26.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:26 smithi123 ceph-mon[34645]: pgmap v598: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 272 MiB/s wr, 1.78k op/s 2024-09-21T03:09:26.972 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:09:26.995 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:26 smithi016 ceph-mon[31253]: pgmap v598: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 272 MiB/s wr, 1.78k op/s 2024-09-21T03:09:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:26 smithi018 ceph-mon[34587]: pgmap v598: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 203 GiB used, 870 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 272 MiB/s wr, 1.78k op/s 2024-09-21T03:09:27.287 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:27.842 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 -- 172.21.15.16:0/2920392466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e01094e0 msgr2=0x7f54e01098f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 --2- 172.21.15.16:0/2920392466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e01094e0 0x7f54e01098f0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009900 tx=0x7f54c802a410 comp rx=0 tx=0).stop 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 -- 172.21.15.16:0/2920392466 shutdown_connections 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 --2- 172.21.15.16:0/2920392466 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e01094e0 0x7f54e01098f0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 --2- 172.21.15.16:0/2920392466 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0109e30 0x7f54e0110660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 --2- 172.21.15.16:0/2920392466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e0103ba0 0x7f54e0103fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 -- 172.21.15.16:0/2920392466 >> 172.21.15.16:0/2920392466 conn(0x7f54e00ff630 msgr2=0x7f54e0101a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 -- 172.21.15.16:0/2920392466 shutdown_connections 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.840+0000 7f54e6250640 1 -- 172.21.15.16:0/2920392466 wait complete. 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.841+0000 7f54e6250640 1 Processor -- start 2024-09-21T03:09:27.843 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.841+0000 7f54e6250640 1 -- start start 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e01094e0 0x7f54e0075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0113af0 con 0x7f54e0109e30 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0113970 con 0x7f54e0103ba0 2024-09-21T03:09:27.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54e6250640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e0113c70 con 0x7f54e01094e0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48512/0 (socket says 172.21.15.16:48512) 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 -- 172.21.15.16:0/3547265896 learned_addr learned my addr 172.21.15.16:0/3547265896 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36764/0 (socket says 172.21.15.16:36764) 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e01094e0 0x7f54e0075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 -- 172.21.15.16:0/3547265896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e01094e0 msgr2=0x7f54e0075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e01094e0 0x7f54e0075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 -- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 msgr2=0x7f54e0076190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54df7fe640 1 -- 172.21.15.16:0/3547265896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e01a87a0 con 0x7f54e0103ba0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.842+0000 7f54dffff640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54df7fe640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9f0 tx=0x7f54d000edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54dd7fa640 1 -- 172.21.15.16:0/3547265896 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d000cd80 con 0x7f54e0103ba0 2024-09-21T03:09:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a8a90 con 0x7f54e0103ba0 2024-09-21T03:09:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54dd7fa640 1 -- 172.21.15.16:0/3547265896 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d00054d0 con 0x7f54e0103ba0 2024-09-21T03:09:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54dd7fa640 1 -- 172.21.15.16:0/3547265896 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0010690 con 0x7f54e0103ba0 2024-09-21T03:09:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.843+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a8fa0 con 0x7f54e0103ba0 2024-09-21T03:09:27.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.846+0000 7f54dd7fa640 1 -- 172.21.15.16:0/3547265896 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f54d0004e90 con 0x7f54e0103ba0 2024-09-21T03:09:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 Processor -- start 2024-09-21T03:09:27.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 -- start start 2024-09-21T03:09:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3e3b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.849 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3e40d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55791c359e50 con 0x55791c03c070 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55791c349280 con 0x55791c0131b0 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55791c34d090 con 0x55791c01b390 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54dd7fa640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 0x7f54cc0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36770/0 (socket says 172.21.15.16:36770) 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54dffff640 1 -- 172.21.15.16:0/3002543898 learned_addr learned my addr 172.21.15.16:0/3002543898 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54dffff640 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3e40d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.847+0000 7f54df7fe640 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3e3b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 0x7f54cc0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 -- 172.21.15.16:0/3002543898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 msgr2=0x55791c3e40d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3e40d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 -- 172.21.15.16:0/3002543898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 msgr2=0x55791c3e3b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3e3b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 -- 172.21.15.16:0/3002543898 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55791c0a7650 con 0x55791c03c070 2024-09-21T03:09:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dffff640 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f54d8013f50 tx=0x7f54d8009f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685c7dd5ff229527 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54dd7fa640 1 -- 172.21.15.16:0/3547265896 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f54d00e7ad0 con 0x7f54e0103ba0 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54d5ffb640 1 -- 172.21.15.16:0/3002543898 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8011d30 con 0x55791c03c070 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 0x7f54cc0b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009230 tx=0x7f54c8031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54d5ffb640 1 -- 172.21.15.16:0/3002543898 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8003e20 con 0x55791c03c070 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.848+0000 7f54d5ffb640 1 -- 172.21.15.16:0/3002543898 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d8020420 con 0x55791c03c070 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- 172.21.15.16:0/3002543898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 msgr2=0x55791c3e3650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f54d8013f50 tx=0x7f54d8009f20 comp rx=0 tx=0).stop 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- 172.21.15.16:0/3002543898 shutdown_connections 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3e40d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3e3b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/3002543898 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3e3650 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- 172.21.15.16:0/3002543898 >> 172.21.15.16:0/3002543898 conn(0x55791c030460 msgr2=0x55791c3e1540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- 172.21.15.16:0/3002543898 shutdown_connections 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- 172.21.15.16:0/3002543898 wait complete. 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 -- start start 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.850+0000 7f54ec7c4740 1 Processor -- start 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3ef2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3fb120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55791c1aa700 con 0x55791c03c070 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55791c2e4110 con 0x55791c0131b0 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55791c3f1e30 con 0x55791c01b390 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54ec7c4740 10 client.0 _ll_register_callbacks cb 0x55791c2fd990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54dffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3fb120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3ef2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48528/0 (socket says 172.21.15.16:48528) 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 -- 172.21.15.16:0/828407104 learned_addr learned my addr 172.21.15.16:0/828407104 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 -- 172.21.15.16:0/828407104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 msgr2=0x55791c3ef2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3ef2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 -- 172.21.15.16:0/828407104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 msgr2=0x55791c3fb120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3fb120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 -- 172.21.15.16:0/828407104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791c311c40 con 0x55791c0131b0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54dffff640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54df7fe640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54d00ad870 tx=0x7f54d00764c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=65a29bb796b19a0a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.851+0000 7f54beffd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0005180 con 0x55791c0131b0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f54d00790d0 con 0x55791c0131b0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d8003e20 con 0x55791c0131b0 2024-09-21T03:09:27.856 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f54d00ad3f0 con 0x55791c0131b0 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 10 client.26024 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55791c3fdbb0 con 0x55791c0131b0 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8003e20 con 0x55791c0131b0 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f54d00ac5b0 con 0x55791c0131b0 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 10 client.26024 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 10 client.26024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 20 client.26024 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791bffa9f0 0x55791c401080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 4 client.26024 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.852+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55791c35bda0 con 0x55791bffa9f0 2024-09-21T03:09:27.857 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.853+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791bffa9f0 0x55791c401080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.853+0000 7f54beffd640 10 client.26024 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.853+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791bffa9f0 0x55791c401080 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7da5cf112ae598b0 server_cookie=7f105bda99bd68e3 in_seq=0 out_seq=0 2024-09-21T03:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54beffd640 1 -- 172.21.15.16:0/828407104 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55791c35bda0 con 0x55791bffa9f0 2024-09-21T03:09:27.858 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54beffd640 10 client.26024 handle_command_reply: tid=0 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 client.26024 shutdown 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 msgr2=0x55791c3eed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f54d00ad870 tx=0x7f54d00764c0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 shutdown_connections 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791bffa9f0 0x55791c401080 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c03c070 0x55791c3fb120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791c01b390 0x55791c3ef2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54beffd640 0 client.26024 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/828407104 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c0131b0 0x55791c3eed80 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 >> 172.21.15.16:0/828407104 conn(0x55791c030460 msgr2=0x55791c3e6830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 shutdown_connections 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 1 -- 172.21.15.16:0/828407104 wait complete. 2024-09-21T03:09:27.859 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.854+0000 7f54ec7c4740 20 client.26024 trim_cache size 0 max 16384 2024-09-21T03:09:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 Processor -- start 2024-09-21T03:09:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 -- start start 2024-09-21T03:09:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.874 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3fb020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3e6750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01afac0 con 0x55791c0131b0 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b2320 con 0x55791c01b390 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b0110 con 0x55791bffa9f0 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54df7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59536/0 (socket says 172.21.15.16:59536) 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54df7fe640 1 -- 172.21.15.16:0/414340800 learned_addr learned my addr 172.21.15.16:0/414340800 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3fb020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.872+0000 7f54dffff640 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3e6750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 -- 172.21.15.16:0/414340800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 msgr2=0x55791c3e6750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3e6750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 -- 172.21.15.16:0/414340800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 msgr2=0x55791c3fb020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.875 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3fb020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 -- 172.21.15.16:0/414340800 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55791c3f0040 con 0x55791bffa9f0 2024-09-21T03:09:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54df7fe640 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54d00ad590 tx=0x7f54d00e5290 comp rx=0 tx=0).ready entity=mon.1 client_cookie=344310b66cf0f129 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54d57fa640 1 -- 172.21.15.16:0/414340800 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d0005180 con 0x55791bffa9f0 2024-09-21T03:09:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54d57fa640 1 -- 172.21.15.16:0/414340800 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0005680 con 0x55791bffa9f0 2024-09-21T03:09:27.876 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.873+0000 7f54d57fa640 1 -- 172.21.15.16:0/414340800 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d00ac5b0 con 0x55791bffa9f0 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 -- 172.21.15.16:0/414340800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 msgr2=0x55791c3e15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f54d00ad590 tx=0x7f54d00e5290 comp rx=0 tx=0).stop 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 -- 172.21.15.16:0/414340800 shutdown_connections 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3fb020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3e6750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/414340800 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3e15b0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 -- 172.21.15.16:0/414340800 >> 172.21.15.16:0/414340800 conn(0x55791c030460 msgr2=0x55791c3e0ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 -- 172.21.15.16:0/414340800 shutdown_connections 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.874+0000 7f54ec7c4740 1 -- 172.21.15.16:0/414340800 wait complete. 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 -- start start 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 Processor -- start 2024-09-21T03:09:27.879 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3f3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3f36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b07e0 con 0x55791c0131b0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b1c50 con 0x55791c01b390 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f54e01b0eb0 con 0x55791bffa9f0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54ec7c4740 10 client.0 _ll_register_callbacks cb 0x7f54d000cf20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54d6ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3f36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54dffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54dffff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48540/0 (socket says 172.21.15.16:48540) 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54dffff640 1 -- 172.21.15.16:0/2223667256 learned_addr learned my addr 172.21.15.16:0/2223667256 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.875+0000 7f54df7fe640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3f3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 -- 172.21.15.16:0/2223667256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 msgr2=0x55791c3f3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3f3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.880 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 -- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 msgr2=0x55791c3f3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 -- 172.21.15.16:0/2223667256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55791c46aa30 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54dffff640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54d6ffd640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3f36a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f54c802a930 tx=0x7f54c802a410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bffc8855b98e43 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54beffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54c8059060 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f54c8004310 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f54d0005680 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f54c8035040 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54ec7c4740 10 client.16395 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.876+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55791c46bd00 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d0005680 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f54c8004ac0 con 0x55791c0131b0 2024-09-21T03:09:27.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 10 client.16395 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 10 client.16395 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 20 client.16395 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791c03c070 0x55791c46e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 4 client.16395 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55791c357850 con 0x55791c03c070 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54df7fe640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791c03c070 0x55791c46e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.877+0000 7f54beffd640 10 client.16395 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:27.882 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54df7fe640 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791c03c070 0x55791c46e420 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=325fca7b944e712b server_cookie=defcaa0bc0103d7 in_seq=0 out_seq=0 2024-09-21T03:09:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54beffd640 1 -- 172.21.15.16:0/2223667256 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4634 (crc 0 0 0) 0x55791c357850 con 0x55791c03c070 2024-09-21T03:09:27.883 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54beffd640 10 client.16395 handle_command_reply: tid=0 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 client.16395 shutdown 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 msgr2=0x55791c3f36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3f36a0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f54c802a930 tx=0x7f54c802a410 comp rx=0 tx=0).stop 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 shutdown_connections 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55791c03c070 0x55791c46e420 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55791c0131b0 0x55791c3f36a0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55791c01b390 0x55791c3f3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 --2- 172.21.15.16:0/2223667256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55791bffa9f0 0x55791c3f3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 >> 172.21.15.16:0/2223667256 conn(0x55791c030460 msgr2=0x55791c3e6750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54beffd640 0 client.16395 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 shutdown_connections 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 1 -- 172.21.15.16:0/2223667256 wait complete. 2024-09-21T03:09:27.884 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.878+0000 7f54ec7c4740 20 client.16395 trim_cache size 0 max 16384 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.885 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.886 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.887 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:09:27.888 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:09:27.889 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:09:27.890 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.891 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.892 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:27.893 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 msgr2=0x7f54cc0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 0x7f54cc0b8c10 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f54c8009230 tx=0x7f54c8031400 comp rx=0 tx=0).stop 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 msgr2=0x7f54e0075710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f54d000e9f0 tx=0x7f54d000edf0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 shutdown_connections 2024-09-21T03:09:27.894 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f54cc0b6700 0x7f54cc0b8c10 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f54e0109e30 0x7f54e0076190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f54e01094e0 0x7f54e0075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 --2- 172.21.15.16:0/3547265896 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f54e0103ba0 0x7f54e0075710 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 >> 172.21.15.16:0/3547265896 conn(0x7f54e00ff630 msgr2=0x7f54e0101a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 shutdown_connections 2024-09-21T03:09:27.895 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:27.879+0000 7f54e6250640 1 -- 172.21.15.16:0/3547265896 wait complete. 2024-09-21T03:09:28.746 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:28 smithi016 ceph-mon[31253]: pgmap v599: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 1.66k op/s 2024-09-21T03:09:28.748 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:09:28.749 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:09:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:28 smithi123 ceph-mon[34645]: pgmap v599: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 1.66k op/s 2024-09-21T03:09:29.070 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:28 smithi018 ceph-mon[34587]: pgmap v599: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 66 GiB data, 202 GiB used, 871 GiB / 1.0 TiB avail; 20 KiB/s rd, 247 MiB/s wr, 1.66k op/s 2024-09-21T03:09:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 -- 172.21.15.16:0/1721885565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 msgr2=0x7f82cc1087d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 --2- 172.21.15.16:0/1721885565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc1087d0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f82c0009900 tx=0x7f82c002a410 comp rx=0 tx=0).stop 2024-09-21T03:09:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 -- 172.21.15.16:0/1721885565 shutdown_connections 2024-09-21T03:09:29.627 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 --2- 172.21.15.16:0/1721885565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc1087d0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 --2- 172.21.15.16:0/1721885565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc108d10 0x7f82cc10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 --2- 172.21.15.16:0/1721885565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc102a80 0x7f82cc102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 -- 172.21.15.16:0/1721885565 >> 172.21.15.16:0/1721885565 conn(0x7f82cc079740 msgr2=0x7f82cc079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 -- 172.21.15.16:0/1721885565 shutdown_connections 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.625+0000 7f82d1029640 1 -- 172.21.15.16:0/1721885565 wait complete. 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 Processor -- start 2024-09-21T03:09:29.628 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 -- start start 2024-09-21T03:09:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 0x7f82cc19a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 0x7f82cc19fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:29.629 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc112980 con 0x7f82cc1083c0 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc112800 con 0x7f82cc108d10 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82d1029640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f82cc112b00 con 0x7f82cc102a80 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82cad9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 0x7f82cc19a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82cb59e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 0x7f82cc19fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.626+0000 7f82ca59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36808/0 (socket says 172.21.15.16:36808) 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 -- 172.21.15.16:0/1221378341 learned_addr learned my addr 172.21.15.16:0/1221378341 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 -- 172.21.15.16:0/1221378341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 msgr2=0x7f82cc19a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 0x7f82cc19a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 -- 172.21.15.16:0/1221378341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 msgr2=0x7f82cc19fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 0x7f82cc19fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 -- 172.21.15.16:0/1221378341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f82cc1a0470 con 0x7f82cc1083c0 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82cad9d640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 0x7f82cc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82cb59e640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 0x7f82cc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:29.630 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82ca59c640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f82c0032fd0 tx=0x7f82c0031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c003f030 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f82cc1a0700 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f82c0004490 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f82c0035070 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.627+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f82cc1a6ec0 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.628+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f82cc103d00 con 0x7f82cc1083c0 2024-09-21T03:09:29.631 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.630+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f82c0042030 con 0x7f82cc1083c0 2024-09-21T03:09:29.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.631+0000 7f82b3fff640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 0x7f82b40b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:29.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.631+0000 7f82cad9d640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 0x7f82b40b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:29.632 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.631+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f82c0107dd0 con 0x7f82cc1083c0 2024-09-21T03:09:29.633 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.632+0000 7f82cad9d640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 0x7f82b40b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009fc0 tx=0x7f82bc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:29.636 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.635+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f82c00d26c0 con 0x7f82cc1083c0 2024-09-21T03:09:29.877 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.876+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f82cc10bb70 con 0x7f82cc1083c0 2024-09-21T03:09:29.878 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.877+0000 7f82b3fff640 1 -- 172.21.15.16:0/1221378341 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f82c00d7600 con 0x7f82cc1083c0 2024-09-21T03:09:29.878 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:09:29.879 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:09:29.879 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 msgr2=0x7f82b40b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 0x7f82b40b8c10 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f82bc009fc0 tx=0x7f82bc005f90 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 msgr2=0x7f82cc19aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f82c0032fd0 tx=0x7f82c0031a20 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 shutdown_connections 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f82b40b6700 0x7f82b40b8c10 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f82cc1083c0 0x7f82cc19aaa0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f82cc108d10 0x7f82cc19fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 --2- 172.21.15.16:0/1221378341 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f82cc102a80 0x7f82cc19a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 >> 172.21.15.16:0/1221378341 conn(0x7f82cc079740 msgr2=0x7f82cc06bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 shutdown_connections 2024-09-21T03:09:29.881 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:29.879+0000 7f82d1029640 1 -- 172.21.15.16:0/1221378341 wait complete. 2024-09-21T03:09:30.714 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:09:30.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:30 smithi123 ceph-mon[34645]: pgmap v600: 97 pgs: 19 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 50 active+clean; 65 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.32k op/s 2024-09-21T03:09:30.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:30 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1221378341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:09:30.981 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:31.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:30 smithi016 ceph-mon[31253]: pgmap v600: 97 pgs: 19 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 50 active+clean; 65 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.32k op/s 2024-09-21T03:09:31.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:30 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1221378341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:09:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:30 smithi018 ceph-mon[34587]: pgmap v600: 97 pgs: 19 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 50 active+clean; 65 GiB data, 200 GiB used, 873 GiB / 1.0 TiB avail; 20 KiB/s rd, 195 MiB/s wr, 1.32k op/s 2024-09-21T03:09:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:30 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1221378341' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:09:31.531 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 -- 172.21.15.16:0/3164505786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0108cf0 msgr2=0x7f18f010f520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 --2- 172.21.15.16:0/3164505786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0108cf0 0x7f18f010f520 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f18e8009ca0 tx=0x7f18e802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 -- 172.21.15.16:0/3164505786 shutdown_connections 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 --2- 172.21.15.16:0/3164505786 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f0108cf0 0x7f18f010f520 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 --2- 172.21.15.16:0/3164505786 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f01083a0 0x7f18f01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 --2- 172.21.15.16:0/3164505786 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f0102e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 -- 172.21.15.16:0/3164505786 >> 172.21.15.16:0/3164505786 conn(0x7f18f0079730 msgr2=0x7f18f0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 -- 172.21.15.16:0/3164505786 shutdown_connections 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.519+0000 7f18f4985640 1 -- 172.21.15.16:0/3164505786 wait complete. 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.520+0000 7f18f4985640 1 Processor -- start 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.520+0000 7f18f4985640 1 -- start start 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.532 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01083a0 0x7f18f019aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f0108cf0 0x7f18f019fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0113a80 con 0x7f18f01083a0 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0113900 con 0x7f18f0102a60 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18f4985640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18f0113c00 con 0x7f18f0108cf0 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48578/0 (socket says 172.21.15.16:48578) 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 -- 172.21.15.16:0/945112618 learned_addr learned my addr 172.21.15.16:0/945112618 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f0108cf0 0x7f18f019fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18edd9b640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01083a0 0x7f18f019aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 -- 172.21.15.16:0/945112618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f0108cf0 msgr2=0x7f18f019fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f0108cf0 0x7f18f019fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 -- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01083a0 msgr2=0x7f18f019aa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01083a0 0x7f18f019aa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 -- 172.21.15.16:0/945112618 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f18f01a0400 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.521+0000 7f18ee59c640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18e000e9f0 tx=0x7f18e000eeb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.522+0000 7f18df7fe640 1 -- 172.21.15.16:0/945112618 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e000cd80 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.522+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18f01a06f0 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.522+0000 7f18df7fe640 1 -- 172.21.15.16:0/945112618 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e00108e0 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.522+0000 7f18df7fe640 1 -- 172.21.15.16:0/945112618 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e0018980 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.522+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18f01a6e50 con 0x7f18f0102a60 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.524+0000 7f18faef9740 1 Processor -- start 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.524+0000 7f18faef9740 1 -- start start 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad85be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d90acfbea0 con 0x55d90a9de070 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d90aceb2d0 con 0x55d90a9bd390 2024-09-21T03:09:31.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18faef9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d90acef0e0 con 0x55d90a9b51b0 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18df7fe640 1 -- 172.21.15.16:0/945112618 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f18e0010430 con 0x7f18f0102a60 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36824/0 (socket says 172.21.15.16:36824) 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18eed9d640 1 -- 172.21.15.16:0/1821897362 learned_addr learned my addr 172.21.15.16:0/1821897362 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18ee59c640 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad85be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 -- 172.21.15.16:0/1821897362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 msgr2=0x55d90ad85be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad85be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 -- 172.21.15.16:0/1821897362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 msgr2=0x55d90ad856a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 -- 172.21.15.16:0/1821897362 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90aa49650 con 0x55d90a9bd390 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18eed9d640 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.525+0000 7f18edd9b640 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18f006aaa0 tx=0x7f18e4011810 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e5d845ff1c59769c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18dcff9640 1 -- 172.21.15.16:0/1821897362 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4009d50 con 0x55d90a9bd390 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18dcff9640 1 -- 172.21.15.16:0/1821897362 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e4005020 con 0x55d90a9bd390 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18dcff9640 1 -- 172.21.15.16:0/1821897362 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4004a00 con 0x55d90a9bd390 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18df7fe640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 0x7f18d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18eed9d640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 0x7f18d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18df7fe640 1 -- 172.21.15.16:0/945112618 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f18e00e7af0 con 0x7f18f0102a60 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18eed9d640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 0x7f18d80b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f18e802f500 tx=0x7f18e802f490 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18faef9740 1 -- 172.21.15.16:0/1821897362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 msgr2=0x55d90ad86120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.526+0000 7f18faef9740 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86120 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f18f006aaa0 tx=0x7f18e4011810 comp rx=0 tx=0).stop 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 -- 172.21.15.16:0/1821897362 shutdown_connections 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86120 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad85be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 --2- 172.21.15.16:0/1821897362 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad856a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.536 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 -- 172.21.15.16:0/1821897362 >> 172.21.15.16:0/1821897362 conn(0x55d90a9d2460 msgr2=0x55d90ad83590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 -- 172.21.15.16:0/1821897362 shutdown_connections 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 -- 172.21.15.16:0/1821897362 wait complete. 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 -- start start 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.527+0000 7f18faef9740 1 Processor -- start 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad90d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55d90ab4c700 con 0x55d90a9de070 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55d90ac86160 con 0x55d90a9bd390 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55d90ad93e80 con 0x55d90a9b51b0 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18faef9740 10 client.0 _ll_register_callbacks cb 0x55d90ac9f9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad90d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48606/0 (socket says 172.21.15.16:48606) 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36828/0 (socket says 172.21.15.16:36828) 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 -- 172.21.15.16:0/3145388741 learned_addr learned my addr 172.21.15.16:0/3145388741 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 -- 172.21.15.16:0/3145388741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 msgr2=0x55d90ad90d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad90d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 -- 172.21.15.16:0/3145388741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 msgr2=0x55d90ad9d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 -- 172.21.15.16:0/3145388741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90acb3c90 con 0x55d90a9bd390 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18edd9b640 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18e40096e0 tx=0x7f18e4005640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=910526cf8e5bc6c7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.528+0000 7f18d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e4029020 con 0x55d90a9bd390 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e4005020 con 0x55d90a9bd390 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f18e40048d0 con 0x55d90a9bd390 2024-09-21T03:09:31.538 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f18e4034050 con 0x55d90a9bd390 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 10 client.26048 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d90ad9fb90 con 0x55d90a9bd390 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e4005020 con 0x55d90a9bd390 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f18e4025c90 con 0x55d90a9bd390 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 10 client.26048 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 10 client.26048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.529+0000 7f18faef9740 20 client.26048 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a99c9f0 0x55d90ada2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18faef9740 4 client.26048 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d90acfddf0 con 0x55d90a99c9f0 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18ee59c640 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a99c9f0 0x55d90ada2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18d5ffb640 10 client.26048 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:31.539 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.530+0000 7f18ee59c640 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a99c9f0 0x55d90ada2fb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35660f112936c7fc server_cookie=f005c60fd60938f7 in_seq=0 out_seq=0 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18d5ffb640 1 -- 172.21.15.16:0/3145388741 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55d90acfddf0 con 0x55d90a99c9f0 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18d5ffb640 10 client.26048 handle_command_reply: tid=0 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 client.26048 shutdown 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 msgr2=0x55d90ad912a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f18e40096e0 tx=0x7f18e4005640 comp rx=0 tx=0).stop 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 shutdown_connections 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a99c9f0 0x55d90ada2fb0 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad912a0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 --2- 172.21.15.16:0/3145388741 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad90d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 >> 172.21.15.16:0/3145388741 conn(0x55d90a9d2460 msgr2=0x55d90ad88880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18d5ffb640 0 client.26048 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:31.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 shutdown_connections 2024-09-21T03:09:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 1 -- 172.21.15.16:0/3145388741 wait complete. 2024-09-21T03:09:31.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.531+0000 7f18faef9740 20 client.26048 trim_cache size 0 max 16384 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.545+0000 7f18faef9740 1 Processor -- start 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.545+0000 7f18faef9740 1 -- start start 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ad7c0 con 0x55d90a9de070 2024-09-21T03:09:31.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01b0020 con 0x55d90a9bd390 2024-09-21T03:09:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18faef9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ade10 con 0x55d90a9b51b0 2024-09-21T03:09:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18eed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59618/0 (socket says 172.21.15.16:59618) 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 -- 172.21.15.16:0/3732015380 learned_addr learned my addr 172.21.15.16:0/3732015380 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18edd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48622/0 (socket says 172.21.15.16:48622) 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 -- 172.21.15.16:0/3732015380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 msgr2=0x55d90ad86de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 -- 172.21.15.16:0/3732015380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 msgr2=0x55d90ad9d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.546+0000 7f18ee59c640 1 -- 172.21.15.16:0/3732015380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90a99cc20 con 0x55d90a9b51b0 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.547+0000 7f18ee59c640 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c6a0 tx=0x7f18e00e9f40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7eae49664df27551 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.547+0000 7f18d7fff640 1 -- 172.21.15.16:0/3732015380 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00ad720 con 0x55d90a9b51b0 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.547+0000 7f18d7fff640 1 -- 172.21.15.16:0/3732015380 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e00eb3f0 con 0x55d90a9b51b0 2024-09-21T03:09:31.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.547+0000 7f18d7fff640 1 -- 172.21.15.16:0/3732015380 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00a90c0 con 0x55d90a9b51b0 2024-09-21T03:09:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 -- 172.21.15.16:0/3732015380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 msgr2=0x55d90ad92c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f18e000c6a0 tx=0x7f18e00e9f40 comp rx=0 tx=0).stop 2024-09-21T03:09:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 -- 172.21.15.16:0/3732015380 shutdown_connections 2024-09-21T03:09:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9de070 0x55d90ad9d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9bd390 0x55d90ad86de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 --2- 172.21.15.16:0/3732015380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9b51b0 0x55d90ad92c40 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 -- 172.21.15.16:0/3732015380 >> 172.21.15.16:0/3732015380 conn(0x55d90a9d2460 msgr2=0x55d90ad866d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 -- 172.21.15.16:0/3732015380 shutdown_connections 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.548+0000 7f18faef9740 1 -- 172.21.15.16:0/3732015380 wait complete. 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 -- start start 2024-09-21T03:09:31.551 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 Processor -- start 2024-09-21T03:09:31.553 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9bd390 0x55d90ad94e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9de070 0x55d90ad95490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01ae4e0 con 0x55d90a9bd390 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01af950 con 0x55d90a9b51b0 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18faef9740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f18f01aebb0 con 0x55d90a9de070 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18ee59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48626/0 (socket says 172.21.15.16:48626) 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18ee59c640 1 -- 172.21.15.16:0/353412177 learned_addr learned my addr 172.21.15.16:0/353412177 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18edd9b640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9bd390 0x55d90ad94e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.549+0000 7f18eed9d640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9de070 0x55d90ad95490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18faef9740 10 client.0 _ll_register_callbacks cb 0x7f18e4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 -- 172.21.15.16:0/353412177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9de070 msgr2=0x55d90ad95490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9de070 0x55d90ad95490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.554 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 -- 172.21.15.16:0/353412177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9bd390 msgr2=0x55d90ad94e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9bd390 0x55d90ad94e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 -- 172.21.15.16:0/353412177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90ad95c00 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18ee59c640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f18e00eb340 tx=0x7f18e0009700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e03e145e1661b553 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18d5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f18e00ad0c0 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f18e00e5420 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f18e00eb3f0 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f18e00ad2a0 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18faef9740 10 client.26054 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.550+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55d90ae0d830 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f18e00eb3f0 con 0x55d90a9b51b0 2024-09-21T03:09:31.555 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f18e00aebd0 con 0x55d90a9b51b0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 10 client.26054 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 10 client.26054 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 20 client.26054 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a971da0 0x55d90ae0ff40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 4 client.26054 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d90acf98a0 con 0x55d90a971da0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.551+0000 7f18edd9b640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a971da0 0x55d90ae0ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18d5ffb640 10 client.26054 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18edd9b640 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a971da0 0x55d90ae0ff40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4545e1a0c8a9a78c server_cookie=74b69b272e2fcfe2 in_seq=0 out_seq=0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18d5ffb640 1 -- 172.21.15.16:0/353412177 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d90acf98a0 con 0x55d90a971da0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18d5ffb640 10 client.26054 handle_command_reply: tid=0 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 client.26054 shutdown 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 msgr2=0x55d90ad99990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f18e00eb340 tx=0x7f18e0009700 comp rx=0 tx=0).stop 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 shutdown_connections 2024-09-21T03:09:31.556 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55d90a971da0 0x55d90ae0ff40 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55d90a9bd390 0x55d90ad94e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55d90a9de070 0x55d90ad95490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18d5ffb640 0 client.26054 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 --2- 172.21.15.16:0/353412177 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55d90a9b51b0 0x55d90ad99990 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 >> 172.21.15.16:0/353412177 conn(0x55d90a9d2460 msgr2=0x55d90ad92a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 shutdown_connections 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 1 -- 172.21.15.16:0/353412177 wait complete. 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.552+0000 7f18faef9740 20 client.26054 trim_cache size 0 max 16384 2024-09-21T03:09:31.557 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 msgr2=0x7f18d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 0x7f18d80b8b90 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f18e802f500 tx=0x7f18e802f490 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 msgr2=0x7f18f019a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f18e000e9f0 tx=0x7f18e000eeb0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 shutdown_connections 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f18d80b6680 0x7f18d80b8b90 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f18f01083a0 0x7f18f019aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18f0108cf0 0x7f18f019fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 --2- 172.21.15.16:0/945112618 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f18f0102a60 0x7f18f019a490 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:31.558 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 >> 172.21.15.16:0/945112618 conn(0x7f18f0079730 msgr2=0x7f18f0104410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:31.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 shutdown_connections 2024-09-21T03:09:31.559 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:31.553+0000 7f18f4985640 1 -- 172.21.15.16:0/945112618 wait complete. 2024-09-21T03:09:32.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:32 smithi123 ceph-mon[34645]: pgmap v601: 97 pgs: 14 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 59 active+clean; 63 GiB data, 194 GiB used, 878 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 690 op/s 2024-09-21T03:09:33.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:32 smithi016 ceph-mon[31253]: pgmap v601: 97 pgs: 14 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 59 active+clean; 63 GiB data, 194 GiB used, 878 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 690 op/s 2024-09-21T03:09:33.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:32 smithi018 ceph-mon[34587]: pgmap v601: 97 pgs: 14 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 59 active+clean; 63 GiB data, 194 GiB used, 878 GiB / 1.0 TiB avail; 40 KiB/s rd, 101 MiB/s wr, 690 op/s 2024-09-21T03:09:33.241 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:09:33.241 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:09:33.514 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.039+0000 7f1f59c41640 1 -- 172.21.15.16:0/163100682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54102a60 msgr2=0x7f1f54102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.039+0000 7f1f59c41640 1 --2- 172.21.15.16:0/163100682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54102a60 0x7f1f54102e70 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f1f44009ca0 tx=0x7f1f4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 -- 172.21.15.16:0/163100682 shutdown_connections 2024-09-21T03:09:34.042 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 --2- 172.21.15.16:0/163100682 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f54108cf0 0x7f1f5410f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 --2- 172.21.15.16:0/163100682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f541083a0 0x7f1f541087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 --2- 172.21.15.16:0/163100682 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54102a60 0x7f1f54102e70 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 -- 172.21.15.16:0/163100682 >> 172.21.15.16:0/163100682 conn(0x7f1f54079730 msgr2=0x7f1f54079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 -- 172.21.15.16:0/163100682 shutdown_connections 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.040+0000 7f1f59c41640 1 -- 172.21.15.16:0/163100682 wait complete. 2024-09-21T03:09:34.043 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 Processor -- start 2024-09-21T03:09:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 -- start start 2024-09-21T03:09:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f54102a60 0x7f1f5419a430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.045 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54108cf0 0x7f1f5419fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54113a80 con 0x7f1f54108cf0 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54113900 con 0x7f1f541083a0 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.041+0000 7f1f59c41640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f54113c00 con 0x7f1f54102a60 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:48644/0 (socket says 172.21.15.16:48644) 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 -- 172.21.15.16:0/1705886167 learned_addr learned my addr 172.21.15.16:0/1705886167 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:34.046 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54108cf0 0x7f1f5419fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f537fe640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f54102a60 0x7f1f5419a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 -- 172.21.15.16:0/1705886167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f54102a60 msgr2=0x7f1f5419a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f54102a60 0x7f1f5419a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 -- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54108cf0 msgr2=0x7f1f5419fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54108cf0 0x7f1f5419fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 -- 172.21.15.16:0/1705886167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f541a03a0 con 0x7f1f541083a0 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00d770 tx=0x7f1f3c00dc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f50ff9640 1 -- 172.21.15.16:0/1705886167 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c017070 con 0x7f1f541083a0 2024-09-21T03:09:34.047 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f541a0690 con 0x7f1f541083a0 2024-09-21T03:09:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f50ff9640 1 -- 172.21.15.16:0/1705886167 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f3c009c30 con 0x7f1f541083a0 2024-09-21T03:09:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f50ff9640 1 -- 172.21.15.16:0/1705886167 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f3c00f620 con 0x7f1f541083a0 2024-09-21T03:09:34.048 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.042+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f541a6df0 con 0x7f1f541083a0 2024-09-21T03:09:34.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 Processor -- start 2024-09-21T03:09:34.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 -- start start 2024-09-21T03:09:34.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f50ff9640 1 -- 172.21.15.16:0/1705886167 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1f3c01a070 con 0x7f1f541083a0 2024-09-21T03:09:34.056 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641a9c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641a9ceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557641a12e60 con 0x5576416f5350 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x557641a02290 con 0x5576416d4670 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f601b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557641a060a0 con 0x5576416cc490 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.045+0000 7f1f53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641a9c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59652/0 (socket says 172.21.15.16:59652) 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 -- 172.21.15.16:0/1175711306 learned_addr learned my addr 172.21.15.16:0/1175711306 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:34.057 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641a9ceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 -- 172.21.15.16:0/1175711306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 msgr2=0x557641a9ceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641a9ceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 -- 172.21.15.16:0/1175711306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 msgr2=0x557641a9c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641a9c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 -- 172.21.15.16:0/1175711306 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557641760930 con 0x5576416cc490 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641a9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f537fe640 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f44035000 tx=0x7f1f440094a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a4784911184de4c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f50ff9640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 0x7f1f400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 0x7f1f400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.046+0000 7f1f49ffb640 1 -- 172.21.15.16:0/1175711306 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44031db0 con 0x5576416cc490 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.047+0000 7f1f49ffb640 1 -- 172.21.15.16:0/1175711306 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f44036030 con 0x5576416cc490 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.047+0000 7f1f49ffb640 1 -- 172.21.15.16:0/1175711306 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44004330 con 0x5576416cc490 2024-09-21T03:09:34.058 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.047+0000 7f1f50ff9640 1 -- 172.21.15.16:0/1705886167 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f1f3c0e7200 con 0x7f1f541083a0 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- 172.21.15.16:0/1175711306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 msgr2=0x557641a9c970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 0x7f1f400b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b910 tx=0x7f1f4c00b2e0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f1f44035000 tx=0x7f1f440094a0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- 172.21.15.16:0/1175711306 shutdown_connections 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641a9ceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416cc490 0x557641a9c970 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1175711306 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641a9c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- 172.21.15.16:0/1175711306 >> 172.21.15.16:0/1175711306 conn(0x5576416e9740 msgr2=0x557641a9a320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- 172.21.15.16:0/1175711306 shutdown_connections 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- 172.21.15.16:0/1175711306 wait complete. 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.048+0000 7f1f601b5740 1 -- start start 2024-09-21T03:09:34.059 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 Processor -- start 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641aa8510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641ab3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x557641863890 con 0x5576416cc490 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55764199cd40 con 0x5576416d4670 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x557641aaaa30 con 0x5576416f5350 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36890/0 (socket says 172.21.15.16:36890) 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f537fe640 1 -- 172.21.15.16:0/2781571394 learned_addr learned my addr 172.21.15.16:0/2781571394 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f53fff640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641ab3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641aa8510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.049+0000 7f1f601b5740 10 client.0 _ll_register_callbacks cb 0x557641a00c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:34.060 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 -- 172.21.15.16:0/2781571394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 msgr2=0x557641ab3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641ab3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 -- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 msgr2=0x557641aa8510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641aa8510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 -- 172.21.15.16:0/2781571394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557641ab42e0 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f53fff640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641ab3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641aa8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f537fe640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f44003f40 tx=0x7f1f4402a8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d054bfa3969375f2 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4404a040 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f44036030 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f1f44031650 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1f4405e050 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f601b5740 10 client.16431 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.050+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557641ab6bf0 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f44055070 con 0x5576416cc490 2024-09-21T03:09:34.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1f4404d030 con 0x5576416cc490 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 10 client.16431 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 10 client.16431 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 20 client.16431 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5576416b3cd0 0x557641ab9f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 4 client.16431 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f1f44036030 con 0x5576416b3cd0 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.051+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5576416b3cd0 0x557641ab9f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.052+0000 7f1f32ffd640 10 client.16431 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.052+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5576416b3cd0 0x557641ab9f50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c8067c75a3937821 server_cookie=a2402da23de9ca72 in_seq=0 out_seq=0 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f32ffd640 1 -- 172.21.15.16:0/2781571394 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x7f1f44036030 con 0x5576416b3cd0 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f32ffd640 10 client.16431 handle_command_reply: tid=0 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 client.16431 shutdown 2024-09-21T03:09:34.062 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 msgr2=0x557641aa7fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f1f44003f40 tx=0x7f1f4402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 shutdown_connections 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5576416b3cd0 0x557641ab9f50 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641ab3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641aa8510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 --2- 172.21.15.16:0/2781571394 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641aa7fd0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 >> 172.21.15.16:0/2781571394 conn(0x5576416e9740 msgr2=0x557641a9f610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f32ffd640 0 client.16431 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 shutdown_connections 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 1 -- 172.21.15.16:0/2781571394 wait complete. 2024-09-21T03:09:34.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.053+0000 7f1f601b5740 20 client.16431 trim_cache size 0 max 16384 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.071+0000 7f1f601b5740 1 Processor -- start 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.071+0000 7f1f601b5740 1 -- start start 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416cc490 0x557641ab3c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416d4670 0x557641aa2550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541ab4d0 con 0x5576416f5350 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541add30 con 0x5576416cc490 2024-09-21T03:09:34.074 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f601b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541abb20 con 0x5576416d4670 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36904/0 (socket says 172.21.15.16:36904) 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 -- 172.21.15.16:0/1703777461 learned_addr learned my addr 172.21.15.16:0/1703777461 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f52ffd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416d4670 0x557641aa2550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f537fe640 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416cc490 0x557641ab3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 -- 172.21.15.16:0/1703777461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416d4670 msgr2=0x557641aa2550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416d4670 0x557641aa2550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 -- 172.21.15.16:0/1703777461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416cc490 msgr2=0x557641ab3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416cc490 0x557641ab3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 -- 172.21.15.16:0/1703777461 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557641aa8d20 con 0x5576416f5350 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.072+0000 7f1f53fff640 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c003f10 tx=0x7f1f4c019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a24b5565f67c5380 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.073+0000 7f1f497fa640 1 -- 172.21.15.16:0/1703777461 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c01ee30 con 0x5576416f5350 2024-09-21T03:09:34.075 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.073+0000 7f1f497fa640 1 -- 172.21.15.16:0/1703777461 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4c0054d0 con 0x5576416f5350 2024-09-21T03:09:34.076 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.073+0000 7f1f497fa640 1 -- 172.21.15.16:0/1703777461 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f4c028680 con 0x5576416f5350 2024-09-21T03:09:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.073+0000 7f1f601b5740 1 -- 172.21.15.16:0/1703777461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 msgr2=0x557641aa4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.073+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c003f10 tx=0x7f1f4c019000 comp rx=0 tx=0).stop 2024-09-21T03:09:34.078 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 -- 172.21.15.16:0/1703777461 shutdown_connections 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416f5350 0x557641aa4fc0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416d4670 0x557641aa2550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 --2- 172.21.15.16:0/1703777461 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416cc490 0x557641ab3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 -- 172.21.15.16:0/1703777461 >> 172.21.15.16:0/1703777461 conn(0x5576416e9740 msgr2=0x557641aa6650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 -- 172.21.15.16:0/1703777461 shutdown_connections 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 -- 172.21.15.16:0/1703777461 wait complete. 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 -- start start 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.074+0000 7f1f601b5740 1 Processor -- start 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641abb520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641abba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.079 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541ac1f0 con 0x5576416cc490 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541ad660 con 0x5576416d4670 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1f541ac8c0 con 0x5576416f5350 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f601b5740 10 client.0 _ll_register_callbacks cb 0x7f1f4c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:36906/0 (socket says 172.21.15.16:36906) 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 -- 172.21.15.16:0/168155291 learned_addr learned my addr 172.21.15.16:0/168155291 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f53fff640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641abba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641abb520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 -- 172.21.15.16:0/168155291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 msgr2=0x557641abba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641abba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 -- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 msgr2=0x557641abb520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641abb520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.080 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 -- 172.21.15.16:0/168155291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557641b23750 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f537fe640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005e30 tx=0x7f1f4402f400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ad4572e89feae2c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.075+0000 7f1f32ffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f44009360 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f1f4402f920 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1f4c0054d0 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1f4402fb20 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 10 client.16437 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x557641b249f0 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f4c0054d0 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1f4404d040 con 0x5576416cc490 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 10 client.16437 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 10 client.16437 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 20 client.16437 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557641689080 0x557641b270b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:34.081 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 4 client.16437 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:09:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.076+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x557641a14db0 con 0x557641689080 2024-09-21T03:09:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.077+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557641689080 0x557641b270b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.077+0000 7f1f52ffd640 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557641689080 0x557641b270b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=703190c14b6897c3 server_cookie=c3552ec78d92f505 in_seq=0 out_seq=0 2024-09-21T03:09:34.082 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.077+0000 7f1f32ffd640 10 client.16437 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f32ffd640 1 -- 172.21.15.16:0/168155291 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x557641a14db0 con 0x557641689080 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f32ffd640 10 client.16437 handle_command_reply: tid=0 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 client.16437 shutdown 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 msgr2=0x557641abafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f1f44005e30 tx=0x7f1f4402f400 comp rx=0 tx=0).stop 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 shutdown_connections 2024-09-21T03:09:34.083 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x557641689080 0x557641b270b0 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5576416f5350 0x557641abba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f32ffd640 0 client.16437 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5576416d4670 0x557641abb520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 --2- 172.21.15.16:0/168155291 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5576416cc490 0x557641abafe0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 >> 172.21.15.16:0/168155291 conn(0x5576416e9740 msgr2=0x557641aa6210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 shutdown_connections 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 1 -- 172.21.15.16:0/168155291 wait complete. 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.079+0000 7f1f601b5740 20 client.16437 trim_cache size 0 max 16384 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "6465f833-9841-4762-b989-30194f9d784e", 2024-09-21T03:09:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:09:34.085 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 msgr2=0x7f1f400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 0x7f1f400b8b90 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f1f4c00b910 tx=0x7f1f4c00b2e0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 msgr2=0x7f1f5419a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f1f3c00d770 tx=0x7f1f3c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 shutdown_connections 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f1f400b6680 0x7f1f400b8b90 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1f54108cf0 0x7f1f5419fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1f541083a0 0x7f1f5419a9d0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 --2- 172.21.15.16:0/1705886167 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1f54102a60 0x7f1f5419a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 >> 172.21.15.16:0/1705886167 conn(0x7f1f54079730 msgr2=0x7f1f54101390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 shutdown_connections 2024-09-21T03:09:34.086 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:34.080+0000 7f1f59c41640 1 -- 172.21.15.16:0/1705886167 wait complete. 2024-09-21T03:09:34.886 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:09:34.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:34 smithi123 ceph-mon[34645]: pgmap v602: 97 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 65 active+clean; 63 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-09-21T03:09:35.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:34 smithi016 ceph-mon[31253]: pgmap v602: 97 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 65 active+clean; 63 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-09-21T03:09:35.153 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:09:35.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:34 smithi018 ceph-mon[34587]: pgmap v602: 97 pgs: 12 active+clean+snaptrim, 20 active+clean+snaptrim_wait, 65 active+clean; 63 GiB data, 193 GiB used, 880 GiB / 1.0 TiB avail; 34 KiB/s rd, 4.7 MiB/s wr, 43 op/s 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 -- 172.21.15.16:0/1513426743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58102ae0 msgr2=0x7f6c58102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/1513426743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58102ae0 0x7f6c58102ef0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c4802a410 comp rx=0 tx=0).stop 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 -- 172.21.15.16:0/1513426743 shutdown_connections 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/1513426743 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58108d20 0x7f6c5810f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/1513426743 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 0x7f6c581087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.651 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/1513426743 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58102ae0 0x7f6c58102ef0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 -- 172.21.15.16:0/1513426743 >> 172.21.15.16:0/1513426743 conn(0x7f6c58079760 msgr2=0x7f6c58079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 -- 172.21.15.16:0/1513426743 shutdown_connections 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.634+0000 7f6c5d69e640 1 -- 172.21.15.16:0/1513426743 wait complete. 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 Processor -- start 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 -- start start 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58102ae0 0x7f6c5819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 0x7f6c5819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.635+0000 7f6c5d69e640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58113ab0 con 0x7f6c58108d20 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c5d69e640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58113930 con 0x7f6c581083d0 2024-09-21T03:09:35.652 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c5d69e640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c58113c30 con 0x7f6c58102ae0 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c567fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 0x7f6c5819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46544/0 (socket says 172.21.15.16:46544) 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 -- 172.21.15.16:0/2688400643 learned_addr learned my addr 172.21.15.16:0/2688400643 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58102ae0 0x7f6c5819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 -- 172.21.15.16:0/2688400643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58102ae0 msgr2=0x7f6c5819a4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58102ae0 0x7f6c5819a4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 -- 172.21.15.16:0/2688400643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 msgr2=0x7f6c5819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 0x7f6c5819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 -- 172.21.15.16:0/2688400643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c581a0450 con 0x7f6c58108d20 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c577fe640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000eaa0 tx=0x7f6c5000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c3ffff640 1 -- 172.21.15.16:0/2688400643 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5000cdb0 con 0x7f6c58108d20 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c581a0740 con 0x7f6c58108d20 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c3ffff640 1 -- 172.21.15.16:0/2688400643 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c50005020 con 0x7f6c58108d20 2024-09-21T03:09:35.653 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.636+0000 7f6c3ffff640 1 -- 172.21.15.16:0/2688400643 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500116d0 con 0x7f6c58108d20 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.637+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c581a6ea0 con 0x7f6c58108d20 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 Processor -- start 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c3ffff640 1 -- 172.21.15.16:0/2688400643 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6c500054a0 con 0x7f6c58108d20 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 -- start start 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 0x55bb240fb650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb24071e50 con 0x55bb23d2b1b0 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bb24061280 con 0x55bb23d54070 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c63c12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb24065090 con 0x55bb23d33390 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46552/0 (socket says 172.21.15.16:46552) 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.640+0000 7f6c56ffd640 1 -- 172.21.15.16:0/2827276872 learned_addr learned my addr 172.21.15.16:0/2827276872 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:35.654 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c567fc640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 0x55bb240fb650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c567fc640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 -- 172.21.15.16:0/2827276872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 msgr2=0x55bb240fc0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 -- 172.21.15.16:0/2827276872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 msgr2=0x55bb240fb650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 0x55bb240fb650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 -- 172.21.15.16:0/2827276872 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb23dbf650 con 0x55bb23d2b1b0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c567fc640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 0x55bb240fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c48032cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e92e7685ae185469 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c3d7fa640 1 -- 172.21.15.16:0/2827276872 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48040070 con 0x55bb23d2b1b0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c3ffff640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 0x7f6c400b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c3d7fa640 1 -- 172.21.15.16:0/2827276872 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c48032e40 con 0x55bb23d2b1b0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c3d7fa640 1 -- 172.21.15.16:0/2827276872 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48035040 con 0x55bb23d2b1b0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c567fc640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 0x7f6c400b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.655 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.641+0000 7f6c3ffff640 1 -- 172.21.15.16:0/2688400643 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f6c500e7800 con 0x7f6c58108d20 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 -- 172.21.15.16:0/2827276872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 msgr2=0x55bb240fbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c567fc640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 0x7f6c400b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c58100c50 tx=0x7f6c4c014040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009900 tx=0x7f6c48032cd0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 -- 172.21.15.16:0/2827276872 shutdown_connections 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb240fbb90 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 --2- 172.21.15.16:0/2827276872 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d54070 0x55bb240fb650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 -- 172.21.15.16:0/2827276872 >> 172.21.15.16:0/2827276872 conn(0x55bb23d48460 msgr2=0x55bb240f9540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 -- 172.21.15.16:0/2827276872 shutdown_connections 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.642+0000 7f6c63c12740 1 -- 172.21.15.16:0/2827276872 wait complete. 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 -- start start 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 Processor -- start 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d2b1b0 0x55bb24106ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb241071e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.656 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55bb23ec2700 con 0x55bb23d54070 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55bb23ffc110 con 0x55bb23d33390 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55bb24109dd0 con 0x55bb23d2b1b0 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c63c12740 10 client.0 _ll_register_callbacks cb 0x55bb24015990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46562/0 (socket says 172.21.15.16:46562) 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 -- 172.21.15.16:0/1529954464 learned_addr learned my addr 172.21.15.16:0/1529954464 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d2b1b0 0x55bb24106ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c567fc640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb241071e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 -- 172.21.15.16:0/1529954464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d2b1b0 msgr2=0x55bb24106ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d2b1b0 0x55bb24106ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 -- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 msgr2=0x55bb241071e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb241071e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.643+0000 7f6c577fe640 1 -- 172.21.15.16:0/1529954464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb24029c40 con 0x55bb23d54070 2024-09-21T03:09:35.657 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c577fe640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ad650 tx=0x7f6c500ace30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f16d9f4ae23c15d server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5001f040 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f6c500acb30 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55bb24073da0 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6c500b7030 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c63c12740 10 client.16449 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb24115aa0 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55bb24073da0 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f6c5000f040 con 0x55bb23d54070 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.644+0000 7f6c63c12740 10 client.16449 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c63c12740 10 client.16449 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c63c12740 20 client.16449 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:35.658 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23d129f0 0x55bb24118f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c63c12740 4 client.16449 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bb2406f850 con 0x55bb23d129f0 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23d129f0 0x55bb24118f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23d129f0 0x55bb24118f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98e7f7dfe6e7af23 server_cookie=c66dbffb04fa4b95 in_seq=0 out_seq=0 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.645+0000 7f6c3aff5640 10 client.16449 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1529954464 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55bb2406f850 con 0x55bb23d129f0 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c3aff5640 10 client.16449 handle_command_reply: tid=0 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 client.16449 shutdown 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 msgr2=0x55bb24107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ad650 tx=0x7f6c500ace30 comp rx=0 tx=0).stop 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 shutdown_connections 2024-09-21T03:09:35.659 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23d129f0 0x55bb24118f70 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb24107720 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb241071e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c3aff5640 0 client.16449 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1529954464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d2b1b0 0x55bb24106ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 >> 172.21.15.16:0/1529954464 conn(0x55bb23d48460 msgr2=0x55bb240fe7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 shutdown_connections 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.646+0000 7f6c63c12740 1 -- 172.21.15.16:0/1529954464 wait complete. 2024-09-21T03:09:35.660 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.647+0000 7f6c63c12740 20 client.16449 trim_cache size 0 max 16384 2024-09-21T03:09:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.664+0000 7f6c63c12740 1 Processor -- start 2024-09-21T03:09:35.666 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.664+0000 7f6c63c12740 1 -- start start 2024-09-21T03:09:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d2b1b0 0x55bb24108b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581ad8d0 con 0x55bb23d54070 2024-09-21T03:09:35.667 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581b0130 con 0x55bb23d2b1b0 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c63c12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581adf20 con 0x55bb23d33390 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46568/0 (socket says 172.21.15.16:46568) 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 -- 172.21.15.16:0/4110286861 learned_addr learned my addr 172.21.15.16:0/4110286861 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c567fc640 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d2b1b0 0x55bb24108b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 -- 172.21.15.16:0/4110286861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 msgr2=0x55bb240fc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 -- 172.21.15.16:0/4110286861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d2b1b0 msgr2=0x55bb24108b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d2b1b0 0x55bb24108b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 -- 172.21.15.16:0/4110286861 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bb24107a60 con 0x55bb23d54070 2024-09-21T03:09:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.665+0000 7f6c577fe640 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ad3c0 tx=0x7f6c500b2270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc0500a9e8cbfd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.666+0000 7f6c3cff9640 1 -- 172.21.15.16:0/4110286861 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c5001f040 con 0x55bb23d54070 2024-09-21T03:09:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.666+0000 7f6c3cff9640 1 -- 172.21.15.16:0/4110286861 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c500b3550 con 0x55bb23d54070 2024-09-21T03:09:35.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.666+0000 7f6c3cff9640 1 -- 172.21.15.16:0/4110286861 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c500ac390 con 0x55bb23d54070 2024-09-21T03:09:35.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- 172.21.15.16:0/4110286861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 msgr2=0x55bb241126f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f6c500ad3c0 tx=0x7f6c500b2270 comp rx=0 tx=0).stop 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- 172.21.15.16:0/4110286861 shutdown_connections 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d54070 0x55bb241126f0 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d33390 0x55bb240fc680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 --2- 172.21.15.16:0/4110286861 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d2b1b0 0x55bb24108b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- 172.21.15.16:0/4110286861 >> 172.21.15.16:0/4110286861 conn(0x55bb23d48460 msgr2=0x55bb24111fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- 172.21.15.16:0/4110286861 shutdown_connections 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- 172.21.15.16:0/4110286861 wait complete. 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 -- start start 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 Processor -- start 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.667+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb2410b5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d54070 0x55bb2410bb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581ae5f0 con 0x55bb23d2b1b0 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581afa60 con 0x55bb23d33390 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6c581aecc0 con 0x55bb23d54070 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c63c12740 10 client.0 _ll_register_callbacks cb 0x7f6c5000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:46578/0 (socket says 172.21.15.16:46578) 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 -- 172.21.15.16:0/1258075208 learned_addr learned my addr 172.21.15.16:0/1258075208 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c577fe640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d54070 0x55bb2410bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c567fc640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb2410b5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 -- 172.21.15.16:0/1258075208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d54070 msgr2=0x55bb2410bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d54070 0x55bb2410bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 -- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 msgr2=0x55bb2410b5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:09:35.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb2410b5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 -- 172.21.15.16:0/1258075208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bb24182910 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c567fc640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb2410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c56ffd640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6c480052d0 tx=0x7f6c48005260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d121532901336bdb server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.668+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c48051070 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(84..84 src has 1..84) ==== 8580+0+0 (secure 0 0 0) 0x7f6c480318b0 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c500b3550 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f6c48035040 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 10 client.16455 fetch_fsmap learned FSMap version 20 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55bb24183ba0 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c500b3550 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f6c48003b50 con 0x55bb23d2b1b0 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 10 client.16455 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 10 client.16455 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:09:35.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 20 client.16455 populate_metadata read hostname 'smithi016' 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23ce7da0 0x55bb24185e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 4 client.16455 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.669+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55bb2406cc80 con 0x55bb23ce7da0 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c567fc640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23ce7da0 0x55bb24185e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c567fc640 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23ce7da0 0x55bb24185e90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a3e76508499901a0 server_cookie=5f6ccdd042c96a7 in_seq=0 out_seq=0 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c3aff5640 10 client.16455 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c3aff5640 1 -- 172.21.15.16:0/1258075208 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5114 (crc 0 0 0) 0x55bb2406cc80 con 0x55bb23ce7da0 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c3aff5640 10 client.16455 handle_command_reply: tid=0 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c63c12740 1 client.16455 shutdown 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 msgr2=0x55bb2410b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.670+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f6c480052d0 tx=0x7f6c48005260 comp rx=0 tx=0).stop 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 shutdown_connections 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55bb23ce7da0 0x55bb24185e90 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55bb23d54070 0x55bb2410bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.676 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c3aff5640 0 client.16455 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55bb23d33390 0x55bb2410b5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 --2- 172.21.15.16:0/1258075208 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55bb23d2b1b0 0x55bb2410b0a0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 >> 172.21.15.16:0/1258075208 conn(0x55bb23d48460 msgr2=0x55bb24108b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 shutdown_connections 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 1 -- 172.21.15.16:0/1258075208 wait complete. 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c63c12740 20 client.16455 trim_cache size 0 max 16384 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:09:35.677 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "6465f833-9841-4762-b989-30194f9d784e", 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.678 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.679 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:09:35.680 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:09:35.681 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.683 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.684 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.685 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:09:35.686 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:09:35.687 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:09:35.688 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:09:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 msgr2=0x7f6c400b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 0x7f6c400b8b90 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f6c58100c50 tx=0x7f6c4c014040 comp rx=0 tx=0).stop 2024-09-21T03:09:35.688 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 msgr2=0x7f6c5819fd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f6c5000eaa0 tx=0x7f6c5000ef60 comp rx=0 tx=0).stop 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 shutdown_connections 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6c400b6680 0x7f6c400b8b90 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6c58108d20 0x7f6c5819fd20 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6c581083d0 0x7f6c5819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 --2- 172.21.15.16:0/2688400643 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6c58102ae0 0x7f6c5819a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.671+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 >> 172.21.15.16:0/2688400643 conn(0x7f6c58079760 msgr2=0x7f6c58101450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.672+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 shutdown_connections 2024-09-21T03:09:35.689 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:09:35.672+0000 7f6c5d69e640 1 -- 172.21.15.16:0/2688400643 wait complete. 2024-09-21T03:09:35.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:35 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:09:35.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:35 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:35.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:35 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:09:35.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:35 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:36.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:35 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:09:36.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:35 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:36.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:35 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:09:36.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:35 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:35 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:09:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:35 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:35 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:09:36.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:35 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:09:36.587 INFO:tasks.cephfs.filesystem:scrub status for tag:6465f833-9841-4762-b989-30194f9d784e - {'path': '/', 'tag': '6465f833-9841-4762-b989-30194f9d784e', 'options': 'recursive,force'} 2024-09-21T03:09:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:36 smithi123 ceph-mon[34645]: pgmap v603: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 61 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 36 KiB/s rd, 786 KiB/s wr, 16 op/s 2024-09-21T03:09:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:36 smithi016 ceph-mon[31253]: pgmap v603: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 61 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 36 KiB/s rd, 786 KiB/s wr, 16 op/s 2024-09-21T03:09:37.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:36 smithi018 ceph-mon[34587]: pgmap v603: 97 pgs: 5 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 82 active+clean; 61 GiB data, 188 GiB used, 885 GiB / 1.0 TiB avail; 36 KiB/s rd, 786 KiB/s wr, 16 op/s 2024-09-21T03:09:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:38 smithi123 ceph-mon[34645]: pgmap v604: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 50 KiB/s rd, 741 KiB/s wr, 18 op/s 2024-09-21T03:09:38.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:38 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:09:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:38 smithi016 ceph-mon[31253]: pgmap v604: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 50 KiB/s rd, 741 KiB/s wr, 18 op/s 2024-09-21T03:09:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:38 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:09:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:38 smithi018 ceph-mon[34587]: pgmap v604: 97 pgs: 3 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 89 active+clean; 61 GiB data, 187 GiB used, 886 GiB / 1.0 TiB avail; 50 KiB/s rd, 741 KiB/s wr, 18 op/s 2024-09-21T03:09:39.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:38 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:09:40.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:40 smithi123 ceph-mon[34645]: pgmap v605: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 535 KiB/s wr, 14 op/s 2024-09-21T03:09:41.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:40 smithi016 ceph-mon[31253]: pgmap v605: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 535 KiB/s wr, 14 op/s 2024-09-21T03:09:41.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:40 smithi018 ceph-mon[34587]: pgmap v605: 97 pgs: 1 active+clean+snaptrim, 96 active+clean; 60 GiB data, 186 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 535 KiB/s wr, 14 op/s 2024-09-21T03:09:42.984 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:42 smithi018 ceph-mon[34587]: pgmap v606: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 193 KiB/s wr, 9 op/s 2024-09-21T03:09:42.984 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:42 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:43.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:42 smithi016 ceph-mon[31253]: pgmap v606: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 193 KiB/s wr, 9 op/s 2024-09-21T03:09:43.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:42 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:42 smithi123 ceph-mon[34645]: pgmap v606: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 37 KiB/s rd, 193 KiB/s wr, 9 op/s 2024-09-21T03:09:43.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:42 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:09:44.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:44.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:44.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:09:44.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:44.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:44.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:09:44.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:09:44.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:09:44.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:09:45.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:45 smithi018 ceph-mon[34587]: pgmap v607: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-09-21T03:09:45.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:45 smithi123 ceph-mon[34645]: pgmap v607: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-09-21T03:09:45.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:45 smithi016 ceph-mon[31253]: pgmap v607: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 5 op/s 2024-09-21T03:09:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:47 smithi018 ceph-mon[34587]: pgmap v608: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-21T03:09:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:47 smithi018 ceph-mon[34587]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:09:47.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:47 smithi123 ceph-mon[34645]: pgmap v608: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-21T03:09:47.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:47 smithi123 ceph-mon[34645]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:09:47.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:47 smithi016 ceph-mon[31253]: pgmap v608: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 17 KiB/s rd, 13 KiB/s wr, 6 op/s 2024-09-21T03:09:47.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:47 smithi016 ceph-mon[31253]: osdmap e85: 12 total, 12 up, 12 in 2024-09-21T03:09:49.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:49 smithi018 ceph-mon[34587]: pgmap v610: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-09-21T03:09:49.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:49 smithi123 ceph-mon[34645]: pgmap v610: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-09-21T03:09:49.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:49 smithi016 ceph-mon[31253]: pgmap v610: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 4.5 KiB/s wr, 3 op/s 2024-09-21T03:09:51.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:51 smithi018 ceph-mon[34587]: pgmap v611: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:51.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:51 smithi123 ceph-mon[34645]: pgmap v611: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:51.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:51 smithi016 ceph-mon[31253]: pgmap v611: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:53 smithi018 ceph-mon[34587]: pgmap v612: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:53.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:53 smithi123 ceph-mon[34645]: pgmap v612: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:53.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:53 smithi016 ceph-mon[31253]: pgmap v612: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:55.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:55 smithi018 ceph-mon[34587]: pgmap v613: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:55.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:55 smithi123 ceph-mon[34645]: pgmap v613: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:55.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:55 smithi016 ceph-mon[31253]: pgmap v613: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:57.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:57 smithi018 ceph-mon[34587]: pgmap v614: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:57.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:57 smithi123 ceph-mon[34645]: pgmap v614: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:57.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:57 smithi016 ceph-mon[31253]: pgmap v614: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 5.5 KiB/s rd, 4.5 KiB/s wr, 2 op/s 2024-09-21T03:09:59.318 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:09:59 smithi016 ceph-mon[31253]: pgmap v615: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 905 B/s rd, 1 op/s 2024-09-21T03:09:59.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:09:59 smithi018 ceph-mon[34587]: pgmap v615: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 905 B/s rd, 1 op/s 2024-09-21T03:09:59.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:09:59 smithi123 ceph-mon[34645]: pgmap v615: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 905 B/s rd, 1 op/s 2024-09-21T03:10:00.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: Health detail: HEALTH_WARN 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: [WRN] BLUESTORE_SLOW_OP_ALERT: 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: osd.2 observed slow operation indications in BlueStore 2024-09-21T03:10:00.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: osd.5 observed slow operation indications in BlueStore 2024-09-21T03:10:00.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: osd.8 observed slow operation indications in BlueStore 2024-09-21T03:10:00.433 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:00 smithi018 ceph-mon[34587]: osd.11 observed slow operation indications in BlueStore 2024-09-21T03:10:00.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: Health detail: HEALTH_WARN 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: [WRN] BLUESTORE_SLOW_OP_ALERT: 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: osd.2 observed slow operation indications in BlueStore 2024-09-21T03:10:00.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: osd.5 observed slow operation indications in BlueStore 2024-09-21T03:10:00.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: osd.8 observed slow operation indications in BlueStore 2024-09-21T03:10:00.472 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:00 smithi123 ceph-mon[34645]: osd.11 observed slow operation indications in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: Health detail: HEALTH_WARN 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: [WRN] BLUESTORE_SLOW_OP_ALERT: 4 OSD(s) experiencing slow operations in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: osd.2 observed slow operation indications in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: osd.5 observed slow operation indications in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: osd.8 observed slow operation indications in BlueStore 2024-09-21T03:10:00.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:00 smithi016 ceph-mon[31253]: osd.11 observed slow operation indications in BlueStore 2024-09-21T03:10:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:01 smithi018 ceph-mon[34587]: pgmap v616: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:01 smithi123 ceph-mon[34645]: pgmap v616: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:01.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:01 smithi016 ceph-mon[31253]: pgmap v616: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:02 smithi018 ceph-mon[34587]: pgmap v617: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:02.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:02 smithi123 ceph-mon[34645]: pgmap v617: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:02.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:02 smithi016 ceph-mon[31253]: pgmap v617: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:04.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:04 smithi016 ceph-mon[31253]: pgmap v618: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:04 smithi018 ceph-mon[34587]: pgmap v618: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:04 smithi123 ceph-mon[34645]: pgmap v618: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 11 KiB/s rd, 22 KiB/s wr, 16 op/s 2024-09-21T03:10:06.588 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:10:06.864 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:06 smithi018 ceph-mon[34587]: pgmap v619: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:06.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:06 smithi123 ceph-mon[34645]: pgmap v619: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:06.985 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:06 smithi016 ceph-mon[31253]: pgmap v619: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 -- 172.21.15.16:0/1079652258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 msgr2=0x7f23501087f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1079652258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f23501087f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f2344009900 tx=0x7f234402a410 comp rx=0 tx=0).stop 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 -- 172.21.15.16:0/1079652258 shutdown_connections 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1079652258 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f23501087f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1079652258 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350108d30 0x7f235010f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1079652258 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350102aa0 0x7f2350102eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 -- 172.21.15.16:0/1079652258 >> 172.21.15.16:0/1079652258 conn(0x7f2350079750 msgr2=0x7f2350079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 -- 172.21.15.16:0/1079652258 shutdown_connections 2024-09-21T03:10:07.400 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.398+0000 7f2354ae6640 1 -- 172.21.15.16:0/1079652258 wait complete. 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 Processor -- start 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 -- start start 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350102aa0 0x7f235019c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350108d30 0x7f23501a1f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23501129f0 con 0x7f23501083e0 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2350112870 con 0x7f2350102aa0 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f2354ae6640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2350112b70 con 0x7f2350108d30 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41820/0 (socket says 172.21.15.16:41820) 2024-09-21T03:10:07.402 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f234dd9b640 1 -- 172.21.15.16:0/1045091242 learned_addr learned my addr 172.21.15.16:0/1045091242 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f234ed9d640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350108d30 0x7f23501a1f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.399+0000 7f234e59c640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350102aa0 0x7f235019c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 -- 172.21.15.16:0/1045091242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350108d30 msgr2=0x7f23501a1f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350108d30 0x7f23501a1f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 -- 172.21.15.16:0/1045091242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350102aa0 msgr2=0x7f235019c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350102aa0 0x7f235019c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 -- 172.21.15.16:0/1045091242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23501a2640 con 0x7f23501083e0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f234dd9b640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f234402a930 tx=0x7f2344031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f233f7fe640 1 -- 172.21.15.16:0/1045091242 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234403f030 con 0x7f23501083e0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f233f7fe640 1 -- 172.21.15.16:0/1045091242 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2344004490 con 0x7f23501083e0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23501a28d0 con 0x7f23501083e0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f233f7fe640 1 -- 172.21.15.16:0/1045091242 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2344035070 con 0x7f23501083e0 2024-09-21T03:10:07.403 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.400+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23501a9090 con 0x7f23501083e0 2024-09-21T03:10:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f233f7fe640 1 -- 172.21.15.16:0/1045091242 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2344042030 con 0x7f23501083e0 2024-09-21T03:10:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 Processor -- start 2024-09-21T03:10:07.404 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 -- start start 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690862650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5646907d8e50 con 0x56469049a390 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5646907c8280 con 0x5646904921b0 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.403+0000 7f235b05a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5646907cc090 con 0x5646904bb070 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.405 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234ed9d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690862650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41822/0 (socket says 172.21.15.16:41822) 2024-09-21T03:10:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234dd9b640 1 -- 172.21.15.16:0/501337906 learned_addr learned my addr 172.21.15.16:0/501337906 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:07.406 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:36162/0 (socket says 172.21.15.16:36162) 2024-09-21T03:10:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f233f7fe640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 0x7f23380b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 -- 172.21.15.16:0/501337906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 msgr2=0x564690862650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.407 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690862650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 -- 172.21.15.16:0/501337906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 msgr2=0x5646908630d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 -- 172.21.15.16:0/501337906 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564690526650 con 0x5646904921b0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234dd9b640 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234ed9d640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 0x7f23380b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f233f7fe640 1 -- 172.21.15.16:0/1045091242 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f23441077e0 con 0x7f23501083e0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.404+0000 7f234e59c640 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2340009b70 tx=0x7f23400117b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=25cd2ff6ecdbec79 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f233cff9640 1 -- 172.21.15.16:0/501337906 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340009e60 con 0x5646904921b0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f233cff9640 1 -- 172.21.15.16:0/501337906 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2340012900 con 0x5646904921b0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f233cff9640 1 -- 172.21.15.16:0/501337906 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234000fa80 con 0x5646904921b0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f235b05a740 1 -- 172.21.15.16:0/501337906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 msgr2=0x564690862b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f234ed9d640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 0x7f23380b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2348005fd0 tx=0x7f2348004a40 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.405+0000 7f235b05a740 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f2340009b70 tx=0x7f23400117b0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.408 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- 172.21.15.16:0/501337906 shutdown_connections 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x56469049a390 0x5646908630d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904921b0 0x564690862b90 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- 172.21.15.16:0/501337906 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690862650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- 172.21.15.16:0/501337906 >> 172.21.15.16:0/501337906 conn(0x5646904af460 msgr2=0x564690860540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- 172.21.15.16:0/501337906 shutdown_connections 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- 172.21.15.16:0/501337906 wait complete. 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- start start 2024-09-21T03:10:07.409 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 Processor -- start 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904921b0 0x56469086dc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 0x56469086e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x564690629700 con 0x5646904bb070 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x564690763110 con 0x56469049a390 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.406+0000 7f235b05a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x564690870e80 con 0x5646904921b0 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f235b05a740 10 client.0 _ll_register_callbacks cb 0x56469077c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 0x56469086e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.412 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234dd9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:36178/0 (socket says 172.21.15.16:36178) 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234dd9b640 1 -- 172.21.15.16:0/566179103 learned_addr learned my addr 172.21.15.16:0/566179103 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234e59c640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904921b0 0x56469086dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 -- 172.21.15.16:0/566179103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904921b0 msgr2=0x56469086dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904921b0 0x56469086dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 -- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 msgr2=0x56469086e190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 -- 172.21.15.16:0/566179103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564690790c40 con 0x5646904bb070 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234dd9b640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f234ed9d640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 0x56469086e6d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2348010a30 tx=0x7f2348004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f0d95908d76e0d5 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f2335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2348011d50 con 0x5646904bb070 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f234801e420 con 0x5646904bb070 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.407+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2340012900 con 0x5646904bb070 2024-09-21T03:10:07.413 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f234801e620 con 0x5646904bb070 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 10 client.16467 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x56469087cb40 con 0x5646904bb070 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2340012900 con 0x5646904bb070 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2348028830 con 0x5646904bb070 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 10 client.16467 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 10 client.16467 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 20 client.16467 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5646904799f0 0x564690880150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 4 client.16467 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.408+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646907dada0 con 0x5646904799f0 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.409+0000 7f234e59c640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5646904799f0 0x564690880150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.409+0000 7f234e59c640 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5646904799f0 0x564690880150 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b3d9278c1c04d83 server_cookie=a6b0a249121cfc46 in_seq=0 out_seq=0 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.409+0000 7f2335ffb640 10 client.16467 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:07.414 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f2335ffb640 1 -- 172.21.15.16:0/566179103 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5646907dada0 con 0x5646904799f0 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f2335ffb640 10 client.16467 handle_command_reply: tid=0 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 client.16467 shutdown 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 msgr2=0x56469086e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 0x56469086e6d0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f2348010a30 tx=0x7f2348004ab0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 shutdown_connections 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5646904799f0 0x564690880150 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904bb070 0x56469086e6d0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x56469086e190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f2335ffb640 0 client.16467 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 --2- 172.21.15.16:0/566179103 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904921b0 0x56469086dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 >> 172.21.15.16:0/566179103 conn(0x5646904af460 msgr2=0x564690864040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 shutdown_connections 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 1 -- 172.21.15.16:0/566179103 wait complete. 2024-09-21T03:10:07.415 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.410+0000 7f235b05a740 20 client.16467 trim_cache size 0 max 16384 2024-09-21T03:10:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 Processor -- start 2024-09-21T03:10:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 -- start start 2024-09-21T03:10:07.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 0x564690868010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 0x56469086ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23501af960 con 0x5646904921b0 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b21c0 con 0x5646904bb070 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f235b05a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23501affb0 con 0x56469049a390 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41840/0 (socket says 172.21.15.16:41840) 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 -- 172.21.15.16:0/3837133623 learned_addr learned my addr 172.21.15.16:0/3837133623 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234ed9d640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 0x56469086ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234dd9b640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 0x564690868010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 -- 172.21.15.16:0/3837133623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 msgr2=0x564690868010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 0x564690868010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 -- 172.21.15.16:0/3837133623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 msgr2=0x56469086ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.434 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 0x56469086ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234e59c640 1 -- 172.21.15.16:0/3837133623 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56469086cbb0 con 0x5646904921b0 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234dd9b640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 0x564690868010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.428+0000 7f234ed9d640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 0x56469086ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f234e59c640 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f234001c040 tx=0x7f234000f700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4218aa49ba7fd31 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f2337fff640 1 -- 172.21.15.16:0/3837133623 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340009770 con 0x5646904921b0 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f2337fff640 1 -- 172.21.15.16:0/3837133623 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234000fc20 con 0x5646904921b0 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f2337fff640 1 -- 172.21.15.16:0/3837133623 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2340005770 con 0x5646904921b0 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f235b05a740 1 -- 172.21.15.16:0/3837133623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 msgr2=0x564690860570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.429+0000 7f235b05a740 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f234001c040 tx=0x7f234000f700 comp rx=0 tx=0).stop 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- 172.21.15.16:0/3837133623 shutdown_connections 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5646904bb070 0x56469086ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x56469049a390 0x564690868010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- 172.21.15.16:0/3837133623 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690860570 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- 172.21.15.16:0/3837133623 >> 172.21.15.16:0/3837133623 conn(0x5646904af460 msgr2=0x56469085fe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- 172.21.15.16:0/3837133623 shutdown_connections 2024-09-21T03:10:07.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- 172.21.15.16:0/3837133623 wait complete. 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- start start 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 Processor -- start 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x564690872540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690872a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0680 con 0x5646904921b0 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b1af0 con 0x56469049a390 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f23501b0d50 con 0x5646904bb070 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.430+0000 7f235b05a740 10 client.0 _ll_register_callbacks cb 0x7f2348009e40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41846/0 (socket says 172.21.15.16:41846) 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 -- 172.21.15.16:0/3813907505 learned_addr learned my addr 172.21.15.16:0/3813907505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234dd9b640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x564690872540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234ed9d640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690872a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 -- 172.21.15.16:0/3813907505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 msgr2=0x564690872a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690872a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 -- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 msgr2=0x564690872540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x564690872540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 -- 172.21.15.16:0/3813907505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646908e9b10 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f234e59c640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f234000ff10 tx=0x7f234000ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48184da646c3b488 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f2335ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f234003b060 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(85..85 src has 1..85) ==== 8536+0+0 (secure 0 0 0) 0x7f234000f980 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.431+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f234000fc20 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f23400050b0 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 10 client.16473 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5646908eac30 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f234000fc20 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f234002d070 con 0x5646904921b0 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 10 client.16473 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 10 client.16473 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:07.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 20 client.16473 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:07.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56469044eda0 0x5646908ed1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:07.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 4 client.16473 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:10:07.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.432+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5646907d6850 con 0x56469044eda0 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f234dd9b640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56469044eda0 0x5646908ed1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f2335ffb640 10 client.16473 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f234dd9b640 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56469044eda0 0x5646908ed1f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7f66e8007c791c92 server_cookie=1bc3fa9ecfe62d18 in_seq=0 out_seq=0 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f2335ffb640 1 -- 172.21.15.16:0/3813907505 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+4918 (crc 0 0 0) 0x5646907d6850 con 0x56469044eda0 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f2335ffb640 10 client.16473 handle_command_reply: tid=0 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f235b05a740 1 client.16473 shutdown 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 msgr2=0x564690872000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.433+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f234000ff10 tx=0x7f234000ff40 comp rx=0 tx=0).stop 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 shutdown_connections 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x56469044eda0 0x5646908ed1f0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5646904bb070 0x564690872a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x56469049a390 0x564690872540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 --2- 172.21.15.16:0/3813907505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5646904921b0 0x564690872000 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2335ffb640 0 client.16473 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 >> 172.21.15.16:0/3813907505 conn(0x5646904af460 msgr2=0x564690863680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 shutdown_connections 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 1 -- 172.21.15.16:0/3813907505 wait complete. 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f235b05a740 20 client.16473 trim_cache size 0 max 16384 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:07.440 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.441 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.442 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:10:07.443 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.444 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.445 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.447 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.448 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:07.449 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 msgr2=0x7f23380b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 0x7f23380b8b90 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f2348005fd0 tx=0x7f2348004a40 comp rx=0 tx=0).stop 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 msgr2=0x7f235019cc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:07.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f234402a930 tx=0x7f2344031a20 comp rx=0 tx=0).stop 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 shutdown_connections 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f23380b6680 0x7f23380b8b90 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f23501083e0 0x7f235019cc70 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2350108d30 0x7f23501a1f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 --2- 172.21.15.16:0/1045091242 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2350102aa0 0x7f235019c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.434+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 >> 172.21.15.16:0/1045091242 conn(0x7f2350079750 msgr2=0x7f2350104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.435+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 shutdown_connections 2024-09-21T03:10:07.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:07.435+0000 7f2354ae6640 1 -- 172.21.15.16:0/1045091242 wait complete. 2024-09-21T03:10:08.365 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:10:08.365 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:10:08.626 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:08.920 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:08 smithi016 ceph-mon[31253]: pgmap v620: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:08 smithi018 ceph-mon[34587]: pgmap v620: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:08.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:08 smithi123 ceph-mon[34645]: pgmap v620: 97 pgs: 97 active+clean; 60 GiB data, 185 GiB used, 887 GiB / 1.0 TiB avail; 12 KiB/s rd, 45 KiB/s wr, 19 op/s 2024-09-21T03:10:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 -- 172.21.15.16:0/2135680059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c1031c0 msgr2=0x7f108c1035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 --2- 172.21.15.16:0/2135680059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c1031c0 0x7f108c1035d0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f1088009900 tx=0x7f108802a410 comp rx=0 tx=0).stop 2024-09-21T03:10:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 -- 172.21.15.16:0/2135680059 shutdown_connections 2024-09-21T03:10:09.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 --2- 172.21.15.16:0/2135680059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c10f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 --2- 172.21.15.16:0/2135680059 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c105190 0x7f108c1055a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 --2- 172.21.15.16:0/2135680059 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c1031c0 0x7f108c1035d0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.229+0000 7f1092fb1640 1 -- 172.21.15.16:0/2135680059 >> 172.21.15.16:0/2135680059 conn(0x7f108c077c60 msgr2=0x7f108c0780a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.230+0000 7f1092fb1640 1 -- 172.21.15.16:0/2135680059 shutdown_connections 2024-09-21T03:10:09.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.230+0000 7f1092fb1640 1 -- 172.21.15.16:0/2135680059 wait complete. 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 Processor -- start 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 -- start start 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c1031c0 0x7f108c115ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c1124c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f108c1118f0 con 0x7f108c105190 2024-09-21T03:10:09.234 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f108c111770 con 0x7f108c1031c0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1092fb1640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f108c111a70 con 0x7f108c105ae0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1083fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41866/0 (socket says 172.21.15.16:41866) 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1083fff640 1 -- 172.21.15.16:0/1663320640 learned_addr learned my addr 172.21.15.16:0/1663320640 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.231+0000 7f1090d4d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c1031c0 0x7f108c115ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f109154e640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c1124c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 -- 172.21.15.16:0/1663320640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 msgr2=0x7f108c1124c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c1124c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 -- 172.21.15.16:0/1663320640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c1031c0 msgr2=0x7f108c115ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c1031c0 0x7f108c115ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 -- 172.21.15.16:0/1663320640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f108c1acc00 con 0x7f108c105190 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f109154e640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c1124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1083fff640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d770 tx=0x7f107c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c017070 con 0x7f108c105190 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f107c009c30 con 0x7f108c105190 2024-09-21T03:10:09.235 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.232+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f108c1ace30 con 0x7f108c105190 2024-09-21T03:10:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.233+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f107c00f620 con 0x7f108c105190 2024-09-21T03:10:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.233+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f108c1ad220 con 0x7f108c105190 2024-09-21T03:10:09.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.235+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f108c1035d0 con 0x7f108c105190 2024-09-21T03:10:09.238 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.236+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f107c01a070 con 0x7f108c105190 2024-09-21T03:10:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.237+0000 7f1081ffb640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 0x7f10740b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.238+0000 7f1090d4d640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 0x7f10740b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:09.239 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.238+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f107c0e72a0 con 0x7f108c105190 2024-09-21T03:10:09.240 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.239+0000 7f1090d4d640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 0x7f10740b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:09.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.244+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f107c009de0 con 0x7f108c105190 2024-09-21T03:10:09.524 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.523+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f108c02e640 con 0x7f108c105190 2024-09-21T03:10:09.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.523+0000 7f1081ffb640 1 -- 172.21.15.16:0/1663320640 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f107c0ad7b0 con 0x7f108c105190 2024-09-21T03:10:09.525 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:10:09.525 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:10:09.526 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 msgr2=0x7f10740b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 0x7f10740b8b90 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f1088000c00 tx=0x7f1088031400 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 msgr2=0x7f108c111da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f107c00d770 tx=0x7f107c00dc30 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 shutdown_connections 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f10740b6680 0x7f10740b8b90 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f108c105190 0x7f108c111da0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f108c105ae0 0x7f108c1124c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 --2- 172.21.15.16:0/1663320640 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f108c1031c0 0x7f108c115ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:09.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 >> 172.21.15.16:0/1663320640 conn(0x7f108c077c60 msgr2=0x7f108c107080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 shutdown_connections 2024-09-21T03:10:09.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:09.526+0000 7f1092fb1640 1 -- 172.21.15.16:0/1663320640 wait complete. 2024-09-21T03:10:09.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:09 smithi018 ceph-mon[34587]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:10:09.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:09 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1663320640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:09.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:09 smithi123 ceph-mon[34645]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:10:09.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:09 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1663320640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:09.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:09 smithi016 ceph-mon[31253]: osdmap e86: 12 total, 12 up, 12 in 2024-09-21T03:10:09.987 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:09 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1663320640' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:10.334 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:10:10.600 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:10.927 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:10 smithi016 ceph-mon[31253]: pgmap v622: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 60 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:10:10.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:10 smithi018 ceph-mon[34587]: pgmap v622: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 60 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:10:10.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:10 smithi123 ceph-mon[34645]: pgmap v622: 97 pgs: 3 active+clean+snaptrim, 6 active+clean+snaptrim_wait, 88 active+clean; 60 GiB data, 185 GiB used, 888 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 -- 172.21.15.16:0/3723418908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 msgr2=0x7f119010f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3723418908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119010f540 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f1188009ca0 tx=0x7f118802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 -- 172.21.15.16:0/3723418908 shutdown_connections 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3723418908 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119010f540 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3723418908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f11901087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3723418908 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f1190102e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.165+0000 7f1195c6f640 1 -- 172.21.15.16:0/3723418908 >> 172.21.15.16:0/3723418908 conn(0x7f1190079740 msgr2=0x7f1190079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.166+0000 7f1195c6f640 1 -- 172.21.15.16:0/3723418908 shutdown_connections 2024-09-21T03:10:11.168 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.166+0000 7f1195c6f640 1 -- 172.21.15.16:0/3723418908 wait complete. 2024-09-21T03:10:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.166+0000 7f1195c6f640 1 Processor -- start 2024-09-21T03:10:11.169 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 -- start start 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119019fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f1190112980 con 0x7f1190108d10 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1190112800 con 0x7f1190102a80 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f1195c6f640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1190112b00 con 0x7f11901083c0 2024-09-21T03:10:11.170 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119019fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119019fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41890/0 (socket says 172.21.15.16:41890) 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118effd640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42888/0 (socket says 172.21.15.16:42888) 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118ffff640 1 -- 172.21.15.16:0/3906530597 learned_addr learned my addr 172.21.15.16:0/3906530597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.167+0000 7f118f7fe640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:36242/0 (socket says 172.21.15.16:36242) 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 -- 172.21.15.16:0/3906530597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 msgr2=0x7f119019aa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 -- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 msgr2=0x7f119019fca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119019fca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 -- 172.21.15.16:0/3906530597 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11901a03d0 con 0x7f1190102a80 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118effd640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118f7fe640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f118000d6d0 tx=0x7f118000db90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118cff9640 1 -- 172.21.15.16:0/3906530597 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180017070 con 0x7f1190102a80 2024-09-21T03:10:11.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118cff9640 1 -- 172.21.15.16:0/3906530597 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1180009820 con 0x7f1190102a80 2024-09-21T03:10:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f118cff9640 1 -- 172.21.15.16:0/3906530597 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1180004c40 con 0x7f1190102a80 2024-09-21T03:10:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11901a06c0 con 0x7f1190102a80 2024-09-21T03:10:11.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.168+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11901a6e20 con 0x7f1190102a80 2024-09-21T03:10:11.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.171+0000 7f119c1e3740 1 Processor -- start 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.171+0000 7f119c1e3740 1 -- start start 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118cff9640 1 -- 172.21.15.16:0/3906530597 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f1180005220 con 0x7f1190102a80 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796d3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796d4120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a879649ea0 con 0x55a8793031b0 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a8796392d0 con 0x55a87930b390 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f119c1e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a87963d0e0 con 0x55a87932c070 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118f7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796d3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42904/0 (socket says 172.21.15.16:42904) 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 -- 172.21.15.16:0/1495899094 learned_addr learned my addr 172.21.15.16:0/1495899094 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118effd640 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796d4120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 -- 172.21.15.16:0/1495899094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 msgr2=0x55a8796d4120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796d4120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 -- 172.21.15.16:0/1495899094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 msgr2=0x55a8796d3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796d3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.172+0000 7f118ffff640 1 -- 172.21.15.16:0/1495899094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a879397650 con 0x55a87932c070 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f118ffff640 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f118802f500 tx=0x7f1188009820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acbc1b4c305a8657 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f1185ffb640 1 -- 172.21.15.16:0/1495899094 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188031db0 con 0x55a87932c070 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f1185ffb640 1 -- 172.21.15.16:0/1495899094 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188036030 con 0x55a87932c070 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f1185ffb640 1 -- 172.21.15.16:0/1495899094 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880042a0 con 0x55a87932c070 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f118cff9640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 0x7f117c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f118f7fe640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 0x7f117c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.173+0000 7f118cff9640 1 -- 172.21.15.16:0/3906530597 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f11800e7200 con 0x7f1190102a80 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- 172.21.15.16:0/1495899094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 msgr2=0x55a8796d36a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f118f7fe640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 0x7f117c0b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11800e49c0 tx=0x7f11800ead50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.178 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f118802f500 tx=0x7f1188009820 comp rx=0 tx=0).stop 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- 172.21.15.16:0/1495899094 shutdown_connections 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796d3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796d4120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1495899094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a87932c070 0x55a8796d36a0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- 172.21.15.16:0/1495899094 >> 172.21.15.16:0/1495899094 conn(0x55a879320460 msgr2=0x55a8796d1590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- 172.21.15.16:0/1495899094 shutdown_connections 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- 172.21.15.16:0/1495899094 wait complete. 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 -- start start 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.174+0000 7f119c1e3740 1 Processor -- start 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8793031b0 0x55a8796decf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 0x55a8796eaf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a87949a700 con 0x55a87932c070 2024-09-21T03:10:11.179 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a8795d4160 con 0x55a87930b390 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a8796e1ca0 con 0x55a8793031b0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f119c1e3740 10 client.0 _ll_register_callbacks cb 0x55a8795ed9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 0x55a8796eaf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:36248/0 (socket says 172.21.15.16:36248) 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 -- 172.21.15.16:0/1296846143 learned_addr learned my addr 172.21.15.16:0/1296846143 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118ffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 0x55a8796eaf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41898/0 (socket says 172.21.15.16:41898) 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118ffff640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8793031b0 0x55a8796decf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 -- 172.21.15.16:0/1296846143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8793031b0 msgr2=0x55a8796decf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8793031b0 0x55a8796decf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 -- 172.21.15.16:0/1296846143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 msgr2=0x55a8796eaf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 0x55a8796eaf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 -- 172.21.15.16:0/1296846143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a879601c90 con 0x55a87930b390 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f118effd640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1178012b10 tx=0x7f117800f3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bed9a7a90bb90474 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.175+0000 7f116effd640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f117800fb40 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f117800fce0 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1188036030 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1178004050 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 10 client.26126 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a8796eda00 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188036030 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1178010430 con 0x55a87930b390 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 10 client.26126 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 10 client.26126 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 20 client.26126 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a8792ea9f0 0x55a8796f0f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 4 client.26126 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:11.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.176+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a87964bdf0 con 0x55a8792ea9f0 2024-09-21T03:10:11.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.177+0000 7f118ffff640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a8792ea9f0 0x55a8796f0f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.177+0000 7f116effd640 10 client.26126 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.177+0000 7f118ffff640 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a8792ea9f0 0x55a8796f0f70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85652de955c9d6 server_cookie=281eed9b6c89b98 in_seq=0 out_seq=0 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f116effd640 1 -- 172.21.15.16:0/1296846143 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a87964bdf0 con 0x55a8792ea9f0 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f116effd640 10 client.26126 handle_command_reply: tid=0 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 client.26126 shutdown 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 msgr2=0x55a8796df230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1178012b10 tx=0x7f117800f3c0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 shutdown_connections 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a8792ea9f0 0x55a8796f0f70 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87932c070 0x55a8796eaf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a8796df230 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f116effd640 0 client.26126 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 --2- 172.21.15.16:0/1296846143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8793031b0 0x55a8796decf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 >> 172.21.15.16:0/1296846143 conn(0x55a879320460 msgr2=0x55a8796d6880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 shutdown_connections 2024-09-21T03:10:11.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 1 -- 172.21.15.16:0/1296846143 wait complete. 2024-09-21T03:10:11.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.178+0000 7f119c1e3740 20 client.26126 trim_cache size 0 max 16384 2024-09-21T03:10:11.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.192+0000 7f119c1e3740 1 Processor -- start 2024-09-21T03:10:11.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.192+0000 7f119c1e3740 1 -- start start 2024-09-21T03:10:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 0x55a8796eae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87930b390 0x55a8796dbee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ad760 con 0x55a87930b390 2024-09-21T03:10:11.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11901affc0 con 0x55a8793031b0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f119c1e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11901addb0 con 0x55a8792ea9f0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118effd640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 0x55a8796eae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:42920/0 (socket says 172.21.15.16:42920) 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 -- 172.21.15.16:0/4119846961 learned_addr learned my addr 172.21.15.16:0/4119846961 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118effd640 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 0x55a8796eae90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:36258/0 (socket says 172.21.15.16:36258) 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87930b390 0x55a8796dbee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 -- 172.21.15.16:0/4119846961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 msgr2=0x55a8796eae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 0x55a8796eae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 -- 172.21.15.16:0/4119846961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87930b390 msgr2=0x55a8796dbee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87930b390 0x55a8796dbee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.193+0000 7f118ffff640 1 -- 172.21.15.16:0/4119846961 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8796d6500 con 0x55a8792ea9f0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.194+0000 7f118ffff640 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1188003dc0 tx=0x7f1188005c50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=783569ed94a6eab3 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.194+0000 7f11857fa640 1 -- 172.21.15.16:0/4119846961 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f118804a040 con 0x55a8792ea9f0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.194+0000 7f11857fa640 1 -- 172.21.15.16:0/4119846961 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188036030 con 0x55a8792ea9f0 2024-09-21T03:10:11.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.194+0000 7f11857fa640 1 -- 172.21.15.16:0/4119846961 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11880042a0 con 0x55a8792ea9f0 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- 172.21.15.16:0/4119846961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 msgr2=0x55a8796df1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f1188003dc0 tx=0x7f1188005c50 comp rx=0 tx=0).stop 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- 172.21.15.16:0/4119846961 shutdown_connections 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a87930b390 0x55a8796dbee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a8793031b0 0x55a8796eae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 --2- 172.21.15.16:0/4119846961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796df1b0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- 172.21.15.16:0/4119846961 >> 172.21.15.16:0/4119846961 conn(0x55a879320460 msgr2=0x55a8796deaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- 172.21.15.16:0/4119846961 shutdown_connections 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- 172.21.15.16:0/4119846961 wait complete. 2024-09-21T03:10:11.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 -- start start 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.195+0000 7f119c1e3740 1 Processor -- start 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796e6ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a879754ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f11901ae480 con 0x55a8793031b0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f11901af8f0 con 0x55a87930b390 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f11901aeb50 con 0x55a8792ea9f0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f119c1e3740 10 client.0 _ll_register_callbacks cb 0x7f1178009cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796e6ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118effd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41910/0 (socket says 172.21.15.16:41910) 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118effd640 1 -- 172.21.15.16:0/630006476 learned_addr learned my addr 172.21.15.16:0/630006476 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a879754ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 -- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 msgr2=0x55a879754ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a879754ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 -- 172.21.15.16:0/630006476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 msgr2=0x55a8796e7430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 -- 172.21.15.16:0/630006476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8797551e0 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118effd640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f118ffff640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796e6ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1188005cc0 tx=0x7f118805a040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84e60ec32410613b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.196+0000 7f116effd640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1188031860 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1188036030 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f118802f660 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f1188004b40 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 10 client.26125 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a879760340 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1188036030 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f1188043070 con 0x55a8792ea9f0 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 10 client.26125 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 10 client.26125 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:11.201 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 20 client.26125 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a87932c070 0x55a879762630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 4 client.26125 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:10:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.197+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8796478a0 con 0x55a87932c070 2024-09-21T03:10:11.202 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f118effd640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a87932c070 0x55a879762630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f118effd640 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a87932c070 0x55a879762630 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d15113d2fe9d3c3 server_cookie=6c58fb105a33bba7 in_seq=0 out_seq=0 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f116effd640 10 client.26125 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f116effd640 1 -- 172.21.15.16:0/630006476 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8796478a0 con 0x55a87932c070 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f116effd640 10 client.26125 handle_command_reply: tid=0 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 client.26125 shutdown 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 msgr2=0x55a8796e6ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796e6ef0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f1188005cc0 tx=0x7f118805a040 comp rx=0 tx=0).stop 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 shutdown_connections 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a87932c070 0x55a879762630 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a8793031b0 0x55a8796e7430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a87930b390 0x55a879754ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.203 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f116effd640 0 client.26125 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 --2- 172.21.15.16:0/630006476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a8792ea9f0 0x55a8796e6ef0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.198+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 >> 172.21.15.16:0/630006476 conn(0x55a879320460 msgr2=0x55a8796d46d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 shutdown_connections 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f119c1e3740 1 -- 172.21.15.16:0/630006476 wait complete. 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f119c1e3740 20 client.26125 trim_cache size 0 max 16384 2024-09-21T03:10:11.204 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 msgr2=0x7f117c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 0x7f117c0b8b90 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f11800e49c0 tx=0x7f11800ead50 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 msgr2=0x7f119019a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f118000d6d0 tx=0x7f118000db90 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 shutdown_connections 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f117c0b6680 0x7f117c0b8b90 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f1190108d10 0x7f119019fca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f11901083c0 0x7f119019aa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 --2- 172.21.15.16:0/3906530597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1190102a80 0x7f119019a460 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 >> 172.21.15.16:0/3906530597 conn(0x7f1190079740 msgr2=0x7f119006be80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 shutdown_connections 2024-09-21T03:10:11.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:11.199+0000 7f1195c6f640 1 -- 172.21.15.16:0/3906530597 wait complete. 2024-09-21T03:10:12.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:12 smithi018 ceph-mon[34587]: pgmap v623: 97 pgs: 17 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 55 active+clean; 59 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:12.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:12 smithi123 ceph-mon[34645]: pgmap v623: 97 pgs: 17 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 55 active+clean; 59 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:13.011 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:10:13.011 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:10:13.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:12 smithi016 ceph-mon[31253]: pgmap v623: 97 pgs: 17 active+clean+snaptrim, 25 active+clean+snaptrim_wait, 55 active+clean; 59 GiB data, 181 GiB used, 892 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:13.275 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 -- 172.21.15.16:0/3805659002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 msgr2=0x7f2efc110620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 --2- 172.21.15.16:0/3805659002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc110620 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4009900 tx=0x7f2ef402a410 comp rx=0 tx=0).stop 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 -- 172.21.15.16:0/3805659002 shutdown_connections 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 --2- 172.21.15.16:0/3805659002 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc110620 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 --2- 172.21.15.16:0/3805659002 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 0x7f2efc1098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 --2- 172.21.15.16:0/3805659002 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 0x7f2efc103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 -- 172.21.15.16:0/3805659002 >> 172.21.15.16:0/3805659002 conn(0x7f2efc07b850 msgr2=0x7f2efc07bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 -- 172.21.15.16:0/3805659002 shutdown_connections 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.809+0000 7f2f03f39640 1 -- 172.21.15.16:0/3805659002 wait complete. 2024-09-21T03:10:13.819 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 Processor -- start 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 -- start start 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 0x7f2efc075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 0x7f2efc075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc113ab0 con 0x7f2efc109df0 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc113930 con 0x7f2efc103b60 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f03f39640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc113c30 con 0x7f2efc1094a0 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41922/0 (socket says 172.21.15.16:41922) 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 -- 172.21.15.16:0/2226477597 learned_addr learned my addr 172.21.15.16:0/2226477597 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 0x7f2efc075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f014d4640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 0x7f2efc075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 -- 172.21.15.16:0/2226477597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 msgr2=0x7f2efc075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.820 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 0x7f2efc075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 -- 172.21.15.16:0/2226477597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 msgr2=0x7f2efc075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 0x7f2efc075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.810+0000 7f2f024d6640 1 -- 172.21.15.16:0/2226477597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2efc1a8700 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2f024d6640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004770 tx=0x7f2ef4004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2efaffd640 1 -- 172.21.15.16:0/2226477597 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef403f030 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2efc1a8990 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2efaffd640 1 -- 172.21.15.16:0/2226477597 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef4004440 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2efaffd640 1 -- 172.21.15.16:0/2226477597 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef4035070 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.811+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2efc1a8e70 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.814+0000 7f2f0a4ad740 1 Processor -- start 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.814+0000 7f2efaffd640 1 -- 172.21.15.16:0/2226477597 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f2ef4042030 con 0x7f2efc109df0 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 -- start start 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 0x55e59b9ac430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.821 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9aceb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e59b922e60 con 0x55e59b5dc490 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e59b912290 con 0x55e59b5e4670 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59b9160a0 con 0x55e59b605350 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f01cd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f01cd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41932/0 (socket says 172.21.15.16:41932) 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f01cd5640 1 -- 172.21.15.16:0/2110924505 learned_addr learned my addr 172.21.15.16:0/2110924505 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f014d4640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 0x55e59b9ac430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.815+0000 7f2f014d4640 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9aceb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2efaffd640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 0x7f2ee40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 -- 172.21.15.16:0/2110924505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 msgr2=0x55e59b9aceb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9aceb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 -- 172.21.15.16:0/2110924505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 msgr2=0x55e59b9ac970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.822 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 -- 172.21.15.16:0/2110924505 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59b670930 con 0x55e59b605350 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f024d6640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 0x7f2ee40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2f014d4640 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 0x55e59b9ac430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009730 tx=0x7f2ef00096c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f44e1282920dedf4 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2efaffd640 1 -- 172.21.15.16:0/2226477597 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f2ef41076c0 con 0x7f2efc109df0 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2ee3fff640 1 -- 172.21.15.16:0/2110924505 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0004050 con 0x55e59b605350 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2ee3fff640 1 -- 172.21.15.16:0/2110924505 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef00054d0 con 0x55e59b605350 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.816+0000 7f2ee3fff640 1 -- 172.21.15.16:0/2110924505 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0004b60 con 0x55e59b605350 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.817+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/2110924505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 msgr2=0x55e59b9ac430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.817+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 0x55e59b9ac430 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009730 tx=0x7f2ef00096c0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.823 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.817+0000 7f2f024d6640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 0x7f2ee40b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ef41053d0 tx=0x7f2ef40c9420 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/2110924505 shutdown_connections 2024-09-21T03:10:13.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5dc490 0x55e59b9ac970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.826 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9aceb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/2110924505 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b605350 0x55e59b9ac430 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/2110924505 >> 172.21.15.16:0/2110924505 conn(0x55e59b5f9740 msgr2=0x55e59b9aa320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/2110924505 shutdown_connections 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/2110924505 wait complete. 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 -- start start 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.818+0000 7f2f0a4ad740 1 Processor -- start 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9b8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b605350 0x55e59b9c3f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55e59b773890 con 0x55e59b5e4670 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55e59b8acd40 con 0x55e59b605350 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55e59b9bac10 con 0x55e59b5dc490 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f0a4ad740 10 client.0 _ll_register_callbacks cb 0x55e59b910c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:13.827 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41948/0 (socket says 172.21.15.16:41948) 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 -- 172.21.15.16:0/3093834109 learned_addr learned my addr 172.21.15.16:0/3093834109 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f014d4640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9b8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f014d4640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b605350 0x55e59b9c3f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 -- 172.21.15.16:0/3093834109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 msgr2=0x55e59b9b8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9b8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 -- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b605350 msgr2=0x55e59b9c3f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b605350 0x55e59b9c3f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f01cd5640 1 -- 172.21.15.16:0/3093834109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59b9c44f0 con 0x55e59b5e4670 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.819+0000 7f2f014d4640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2eec011d40 tx=0x7f2eec00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc7f73e555295318 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec004b20 con 0x55e59b5e4670 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f2eec004cc0 con 0x55e59b5e4670 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef00054d0 con 0x55e59b5e4670 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2eec003970 con 0x55e59b5e4670 2024-09-21T03:10:13.828 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 10 client.16509 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e59b9c6e00 con 0x55e59b5e4670 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef00054d0 con 0x55e59b5e4670 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2eec019070 con 0x55e59b5e4670 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 10 client.16509 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 10 client.16509 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 20 client.16509 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b5c3cd0 0x55e59b9ca3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 4 client.16509 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.820+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e59b924db0 con 0x55e59b5c3cd0 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.821+0000 7f2f014d4640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b5c3cd0 0x55e59b9ca3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.821+0000 7f2ee17fa640 10 client.16509 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.821+0000 7f2f014d4640 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b5c3cd0 0x55e59b9ca3a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=72e26c7512f4f6e8 server_cookie=c90ccefbaba1a45 in_seq=0 out_seq=0 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2ee17fa640 1 -- 172.21.15.16:0/3093834109 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55e59b924db0 con 0x55e59b5c3cd0 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2ee17fa640 10 client.16509 handle_command_reply: tid=0 2024-09-21T03:10:13.829 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 client.16509 shutdown 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 msgr2=0x55e59b9b8570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f2eec011d40 tx=0x7f2eec00f790 comp rx=0 tx=0).stop 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 shutdown_connections 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b5c3cd0 0x55e59b9ca3a0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b5e4670 0x55e59b9b8570 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b605350 0x55e59b9c3f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2ee17fa640 0 client.16509 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/3093834109 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9b8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 >> 172.21.15.16:0/3093834109 conn(0x55e59b5f9740 msgr2=0x55e59b9af5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 shutdown_connections 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/3093834109 wait complete. 2024-09-21T03:10:13.830 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.822+0000 7f2f0a4ad740 20 client.16509 trim_cache size 0 max 16384 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 Processor -- start 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 -- start start 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5dc490 0x55e59b9b35e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5e4670 0x55e59b9c3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1af9c0 con 0x55e59b605350 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1b2220 con 0x55e59b5dc490 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1b0010 con 0x55e59b5e4670 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41950/0 (socket says 172.21.15.16:41950) 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 -- 172.21.15.16:0/1035328893 learned_addr learned my addr 172.21.15.16:0/1035328893 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5e4670 0x55e59b9c3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.844 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5dc490 0x55e59b9b35e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 -- 172.21.15.16:0/1035328893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5e4670 msgr2=0x55e59b9c3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5e4670 0x55e59b9c3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 -- 172.21.15.16:0/1035328893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5dc490 msgr2=0x55e59b9b35e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5dc490 0x55e59b9b35e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.842+0000 7f2f014d4640 1 -- 172.21.15.16:0/1035328893 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e59b9ae720 con 0x55e59b605350 2024-09-21T03:10:13.845 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.843+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009730 tx=0x7f2ef0005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cba538e5e9e385dd server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.843+0000 7f2ee37fe640 1 -- 172.21.15.16:0/1035328893 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef0011050 con 0x55e59b605350 2024-09-21T03:10:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.843+0000 7f2ee37fe640 1 -- 172.21.15.16:0/1035328893 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef0005410 con 0x55e59b605350 2024-09-21T03:10:13.846 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.843+0000 7f2ee37fe640 1 -- 172.21.15.16:0/1035328893 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2ef000fcf0 con 0x55e59b605350 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1035328893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 msgr2=0x55e59b9ade20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f2ef0009730 tx=0x7f2ef0005f90 comp rx=0 tx=0).stop 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1035328893 shutdown_connections 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59b9ade20 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5e4670 0x55e59b9c3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1035328893 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5dc490 0x55e59b9b35e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1035328893 >> 172.21.15.16:0/1035328893 conn(0x55e59b5f9740 msgr2=0x55e59b9b9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1035328893 shutdown_connections 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1035328893 wait complete. 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 -- start start 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.844+0000 7f2f0a4ad740 1 Processor -- start 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9bc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.847 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9bc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1b06e0 con 0x55e59b605350 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1b1b50 con 0x55e59b5e4670 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2efc1b0db0 con 0x55e59b5dc490 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f0a4ad740 10 client.0 _ll_register_callbacks cb 0x7f2eec009ce0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f014d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f014d4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:41956/0 (socket says 172.21.15.16:41956) 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f014d4640 1 -- 172.21.15.16:0/1039070339 learned_addr learned my addr 172.21.15.16:0/1039070339 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9bc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.848 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9bc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 -- 172.21.15.16:0/1039070339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 msgr2=0x55e59b9bc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9bc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 -- 172.21.15.16:0/1039070339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 msgr2=0x55e59ba33d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f01cd5640 1 -- 172.21.15.16:0/1039070339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e59ba34280 con 0x55e59b5e4670 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.845+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2f01cd5640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9bc840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eec01a500 tx=0x7f2eec00fc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=85250b357173f215 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2ee17fa640 10 client.0 ms_handle_connect on v2:172.21.15.18:3300/0 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2eec01a900 con 0x55e59b5e4670 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2ef0005410 con 0x55e59b5e4670 2024-09-21T03:10:13.850 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mon.2 v2:172.21.15.18:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f2eec01aaa0 con 0x55e59b5e4670 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f2eec038050 con 0x55e59b5e4670 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2f0a4ad740 10 client.26150 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55e59ba35440 con 0x55e59b5e4670 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.846+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mon.2 v2:172.21.15.18:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2ef0005410 con 0x55e59b5e4670 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mon.2 v2:172.21.15.18:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f2eec019040 con 0x55e59b5e4670 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 10 client.26150 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 10 client.26150 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 20 client.26150 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b599080 0x55e59ba37b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 4 client.26150 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55e59b920860 con 0x55e59b599080 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b599080 0x55e59ba37b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2ee17fa640 10 client.26150 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:13.851 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.847+0000 7f2f014d4640 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b599080 0x55e59ba37b90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2517f399fa94e1dd server_cookie=a4a9702ccd63e874 in_seq=0 out_seq=0 2024-09-21T03:10:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2ee17fa640 1 -- 172.21.15.16:0/1039070339 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55e59b920860 con 0x55e59b599080 2024-09-21T03:10:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2ee17fa640 10 client.26150 handle_command_reply: tid=0 2024-09-21T03:10:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 client.26150 shutdown 2024-09-21T03:10:13.852 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 msgr2=0x55e59b9bc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9bc840 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f2eec01a500 tx=0x7f2eec00fc90 comp rx=0 tx=0).stop 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 shutdown_connections 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55e59b599080 0x55e59ba37b90 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55e59b605350 0x55e59ba33d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55e59b5e4670 0x55e59b9bc840 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2ee17fa640 0 client.26150 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 --2- 172.21.15.16:0/1039070339 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55e59b5dc490 0x55e59b9bc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 >> 172.21.15.16:0/1039070339 conn(0x55e59b5f9740 msgr2=0x55e59b9b9ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 shutdown_connections 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 1 -- 172.21.15.16:0/1039070339 wait complete. 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.849+0000 7f2f0a4ad740 20 client.26150 trim_cache size 0 max 16384 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:13.853 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "749c373e-c09d-4335-b4aa-d8d285c516ab", 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 msgr2=0x7f2ee40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 0x7f2ee40b8b90 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f2ef41053d0 tx=0x7f2ef40c9420 comp rx=0 tx=0).stop 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 msgr2=0x7f2efc076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:13.854 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f2ef4004770 tx=0x7f2ef4004040 comp rx=0 tx=0).stop 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 shutdown_connections 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f2ee40b6680 0x7f2ee40b8b90 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2efc109df0 0x7f2efc076190 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2efc1094a0 0x7f2efc075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 --2- 172.21.15.16:0/2226477597 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2efc103b60 0x7f2efc075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 >> 172.21.15.16:0/2226477597 conn(0x7f2efc07b850 msgr2=0x7f2efc105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 shutdown_connections 2024-09-21T03:10:13.855 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:13.850+0000 7f2f03f39640 1 -- 172.21.15.16:0/2226477597 wait complete. 2024-09-21T03:10:14.545 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:10:14.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:14 smithi016 ceph-mon[31253]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 58 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:14 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:10:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:14 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:14.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:14 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:10:14.856 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:14 smithi018 ceph-mon[34587]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 58 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:14 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:10:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:14 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:14.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:14 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:10:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:14 smithi123 ceph-mon[34645]: pgmap v624: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 58 GiB data, 179 GiB used, 894 GiB / 1.0 TiB avail; 2.4 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:10:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:14 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:10:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:14 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:14.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:14 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 -- 172.21.15.16:0/2655095712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c104a30 msgr2=0x7f634c104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 --2- 172.21.15.16:0/2655095712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c104a30 0x7f634c104e40 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f633c009900 tx=0x7f633c02a410 comp rx=0 tx=0).stop 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 -- 172.21.15.16:0/2655095712 shutdown_connections 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 --2- 172.21.15.16:0/2655095712 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c10de60 0x7f634c110330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 --2- 172.21.15.16:0/2655095712 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c105380 0x7f634c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 --2- 172.21.15.16:0/2655095712 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c104a30 0x7f634c104e40 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.505 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 -- 172.21.15.16:0/2655095712 >> 172.21.15.16:0/2655095712 conn(0x7f634c079760 msgr2=0x7f634c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 -- 172.21.15.16:0/2655095712 shutdown_connections 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.447+0000 7f6352639640 1 -- 172.21.15.16:0/2655095712 wait complete. 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.448+0000 7f6352639640 1 Processor -- start 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.448+0000 7f6352639640 1 -- start start 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 0x7f634c102770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c10de60 0x7f634c1031f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c113ac0 con 0x7f634c105380 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f634c113940 con 0x7f634c104a30 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f634c113c40 con 0x7f634c10de60 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59232/0 (socket says 172.21.15.16:59232) 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 -- 172.21.15.16:0/1193201870 learned_addr learned my addr 172.21.15.16:0/1193201870 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:15.506 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6350bd6640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c10de60 0x7f634c1031f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634bfff640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 0x7f634c102770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 -- 172.21.15.16:0/1193201870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c10de60 msgr2=0x7f634c1031f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c10de60 0x7f634c1031f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 -- 172.21.15.16:0/1193201870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 msgr2=0x7f634c102770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 0x7f634c102770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 -- 172.21.15.16:0/1193201870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634c1a8720 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634bfff640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 0x7f634c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f634b7fe640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f634000e9c0 tx=0x7f634000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f63497fa640 1 -- 172.21.15.16:0/1193201870 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f634000ccc0 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f63497fa640 1 -- 172.21.15.16:0/1193201870 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63400054d0 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f634c1a8a10 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.449+0000 7f63497fa640 1 -- 172.21.15.16:0/1193201870 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340010430 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.450+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f634c1a8f00 con 0x7f634c105380 2024-09-21T03:10:15.507 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 Processor -- start 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 -- start start 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f63497fa640 1 -- 172.21.15.16:0/1193201870 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6340005020 con 0x7f634c105380 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff3ffb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff037390 0x5642ff4000d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642ff375e50 con 0x5642ff037390 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5642ff365280 con 0x5642ff02f1b0 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.452+0000 7f6358bad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642ff369090 con 0x5642ff058070 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff037390 0x5642ff4000d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:59758/0 (socket says 172.21.15.16:59758) 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 -- 172.21.15.16:0/1894581467 learned_addr learned my addr 172.21.15.16:0/1894581467 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:15.508 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f634bfff640 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff3ffb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 -- 172.21.15.16:0/1894581467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 msgr2=0x5642ff3ffb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff3ffb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 -- 172.21.15.16:0/1894581467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff037390 msgr2=0x5642ff4000d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff037390 0x5642ff4000d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 -- 172.21.15.16:0/1894581467 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ff0c3650 con 0x5642ff058070 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f634bfff640 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff3ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f63497fa640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 0x7f63340b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f6350bd6640 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6344007f40 tx=0x7f634400f850 comp rx=0 tx=0).ready entity=mon.1 client_cookie=84a6269e5dcc5747 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f634bfff640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 0x7f63340b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f63497fa640 1 -- 172.21.15.16:0/1193201870 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f63400e7b90 con 0x7f634c105380 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f63327fc640 1 -- 172.21.15.16:0/1894581467 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344010900 con 0x5642ff058070 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f63327fc640 1 -- 172.21.15.16:0/1894581467 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63440183f0 con 0x5642ff058070 2024-09-21T03:10:15.509 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.453+0000 7f63327fc640 1 -- 172.21.15.16:0/1894581467 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344017600 con 0x5642ff058070 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 -- 172.21.15.16:0/1894581467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 msgr2=0x5642ff3ff650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f634bfff640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 0x7f63340b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f633c009230 tx=0x7f633c005e60 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f6344007f40 tx=0x7f634400f850 comp rx=0 tx=0).stop 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 -- 172.21.15.16:0/1894581467 shutdown_connections 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff037390 0x5642ff4000d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff3ffb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 --2- 172.21.15.16:0/1894581467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff058070 0x5642ff3ff650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 -- 172.21.15.16:0/1894581467 >> 172.21.15.16:0/1894581467 conn(0x5642ff04c460 msgr2=0x5642ff3fd540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 -- 172.21.15.16:0/1894581467 shutdown_connections 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.454+0000 7f6358bad740 1 -- 172.21.15.16:0/1894581467 wait complete. 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 -- start start 2024-09-21T03:10:15.510 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 Processor -- start 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x5642ff1c6700 con 0x5642ff058070 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x5642ff300110 con 0x5642ff02f1b0 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5642ff40de30 con 0x5642ff037390 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.455+0000 7f6358bad740 10 client.0 _ll_register_callbacks cb 0x5642ff319990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59260/0 (socket says 172.21.15.16:59260) 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f6350bd6640 1 -- 172.21.15.16:0/2932271325 learned_addr learned my addr 172.21.15.16:0/2932271325 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634bfff640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.511 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 -- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 msgr2=0x5642ff40acb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40acb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 -- 172.21.15.16:0/2932271325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 msgr2=0x5642ff417120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 -- 172.21.15.16:0/2932271325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ff32dc40 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f6350bd6640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634bfff640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f634b7fe640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40b1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63400e52a0 tx=0x7f63400eb2d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=997f7ecf9cc1811f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f632b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400ad770 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f63400a92f0 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.456+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63440183f0 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f634000ce60 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 10 client.26155 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5642ff419ab0 con 0x5642ff037390 2024-09-21T03:10:15.512 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63440183f0 con 0x5642ff037390 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f63400f0070 con 0x5642ff037390 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 10 client.26155 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 10 client.26155 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 20 client.26155 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642ff0169f0 0x5642ff41cf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 4 client.26155 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642ff377da0 con 0x5642ff0169f0 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.457+0000 7f634bfff640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642ff0169f0 0x5642ff41cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.458+0000 7f634bfff640 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642ff0169f0 0x5642ff41cf70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f2d2e7fa1df6d1 server_cookie=4a178bd8775f88ae in_seq=0 out_seq=0 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.458+0000 7f632b7fe640 10 client.26155 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f632b7fe640 1 -- 172.21.15.16:0/2932271325 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x5642ff377da0 con 0x5642ff0169f0 2024-09-21T03:10:15.513 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f632b7fe640 10 client.26155 handle_command_reply: tid=0 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 client.26155 shutdown 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 msgr2=0x5642ff40b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40b1f0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f63400e52a0 tx=0x7f63400eb2d0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 shutdown_connections 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642ff0169f0 0x5642ff41cf70 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff417120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f632b7fe640 0 client.26155 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40b1f0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 --2- 172.21.15.16:0/2932271325 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 >> 172.21.15.16:0/2932271325 conn(0x5642ff04c460 msgr2=0x5642ff4027e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 shutdown_connections 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 1 -- 172.21.15.16:0/2932271325 wait complete. 2024-09-21T03:10:15.514 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.459+0000 7f6358bad740 20 client.26155 trim_cache size 0 max 16384 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 Processor -- start 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 -- start start 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40cca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff3fcb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1af9e0 con 0x5642ff058070 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b2240 con 0x5642ff02f1b0 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6358bad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b0030 con 0x5642ff037390 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff3fcb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59268/0 (socket says 172.21.15.16:59268) 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 -- 172.21.15.16:0/3675219590 learned_addr learned my addr 172.21.15.16:0/3675219590 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:15.515 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f634bfff640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 -- 172.21.15.16:0/3675219590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 msgr2=0x5642ff3fcb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff3fcb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.479+0000 7f6350bd6640 1 -- 172.21.15.16:0/3675219590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 msgr2=0x5642ff40cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6350bd6640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6350bd6640 1 -- 172.21.15.16:0/3675219590 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642ff4024b0 con 0x5642ff058070 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f634bfff640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f634b7fe640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff3fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6350bd6640 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6344007f40 tx=0x7f6344019ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d6d95ed556cc91 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6331ffb640 1 -- 172.21.15.16:0/3675219590 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344003d00 con 0x5642ff058070 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6331ffb640 1 -- 172.21.15.16:0/3675219590 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6344007940 con 0x5642ff058070 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.480+0000 7f6331ffb640 1 -- 172.21.15.16:0/3675219590 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6344018750 con 0x5642ff058070 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- 172.21.15.16:0/3675219590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 msgr2=0x5642ff3fd0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7f6344007f40 tx=0x7f6344019ea0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- 172.21.15.16:0/3675219590 shutdown_connections 2024-09-21T03:10:15.516 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff3fd0b0 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff3fcb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 --2- 172.21.15.16:0/3675219590 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- 172.21.15.16:0/3675219590 >> 172.21.15.16:0/3675219590 conn(0x5642ff04c460 msgr2=0x5642ff401040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- 172.21.15.16:0/3675219590 shutdown_connections 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- 172.21.15.16:0/3675219590 wait complete. 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.481+0000 7f6358bad740 1 -- start start 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 Processor -- start 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40f0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b0700 con 0x5642ff058070 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b1b70 con 0x5642ff02f1b0 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f634c1b0dd0 con 0x5642ff037390 2024-09-21T03:10:15.517 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6358bad740 10 client.0 _ll_register_callbacks cb 0x7f63400eb3d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634bfff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40f0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6350bd6640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:59272/0 (socket says 172.21.15.16:59272) 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6350bd6640 1 -- 172.21.15.16:0/693872222 learned_addr learned my addr 172.21.15.16:0/693872222 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 -- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 msgr2=0x5642ff40f0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40f0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 -- 172.21.15.16:0/693872222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 msgr2=0x5642ff40fb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634b7fe640 1 -- 172.21.15.16:0/693872222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642ff486920 con 0x5642ff037390 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f634bfff640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.482+0000 7f6350bd6640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f634b7fe640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40f620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63400b7580 tx=0x7f6340005f10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=338bc9076e47aad7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:10:15.518 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63400ad110 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6344007940 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(86..86 src has 1..86) ==== 8580+0+0 (secure 0 0 0) 0x7f63400ac670 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f63400ad2f0 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f6358bad740 10 client.26161 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5642ff487bb0 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6344007940 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.483+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f63400f0040 con 0x5642ff037390 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 10 client.26161 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 10 client.26161 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 20 client.26161 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642fefebda0 0x5642ff48a230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 4 client.26161 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5642ff373850 con 0x5642fefebda0 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f634bfff640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642fefebda0 0x5642ff48a230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:15.519 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f634bfff640 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642fefebda0 0x5642ff48a230 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=182a479c1edde188 server_cookie=c119d4cb895efbe3 in_seq=0 out_seq=0 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f632b7fe640 10 client.26161 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f632b7fe640 1 -- 172.21.15.16:0/693872222 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5398 (crc 0 0 0) 0x5642ff373850 con 0x5642fefebda0 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f632b7fe640 10 client.26161 handle_command_reply: tid=0 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.484+0000 7f6358bad740 1 client.26161 shutdown 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stdout: "749c373e-c09d-4335-b4aa-d8d285c516ab": { 2024-09-21T03:10:15.520 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "749c373e-c09d-4335-b4aa-d8d285c516ab", 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "749c373e-c09d-4335-b4aa-d8d285c516ab": { 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:13.847478+0000", 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.521 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:10:15.522 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.523 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.524 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:10:15.525 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:10:15.526 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.528 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.529 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.530 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.531 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:15.532 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 msgr2=0x5642ff40f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40f620 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63400b7580 tx=0x7f6340005f10 comp rx=0 tx=0).stop 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 shutdown_connections 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5642fefebda0 0x5642ff48a230 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5642ff058070 0x5642ff40fb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5642ff037390 0x5642ff40f620 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 --2- 172.21.15.16:0/693872222 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5642ff02f1b0 0x5642ff40f0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.533 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 >> 172.21.15.16:0/693872222 conn(0x5642ff04c460 msgr2=0x5642ff400e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f632b7fe640 0 client.26161 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 shutdown_connections 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 1 -- 172.21.15.16:0/693872222 wait complete. 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6358bad740 20 client.26161 trim_cache size 0 max 16384 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 msgr2=0x7f63340b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 0x7f63340b8b90 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f633c009230 tx=0x7f633c005e60 comp rx=0 tx=0).stop 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 msgr2=0x7f634c102cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.485+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7f634000e9c0 tx=0x7f634000ee80 comp rx=0 tx=0).stop 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 shutdown_connections 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f63340b6680 0x7f63340b8b90 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f634c105380 0x7f634c102cb0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f634c10de60 0x7f634c1031f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.534 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 --2- 172.21.15.16:0/1193201870 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f634c104a30 0x7f634c102770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 >> 172.21.15.16:0/1193201870 conn(0x7f634c079760 msgr2=0x7f634c1008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 shutdown_connections 2024-09-21T03:10:15.535 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:15.486+0000 7f6352639640 1 -- 172.21.15.16:0/1193201870 wait complete. 2024-09-21T03:10:15.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:15 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:15.959 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:15 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:15.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:15 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:16.288 INFO:tasks.cephfs.filesystem:scrub status for tag:749c373e-c09d-4335-b4aa-d8d285c516ab - {'path': '/', 'tag': '749c373e-c09d-4335-b4aa-d8d285c516ab', 'options': 'recursive,force'} 2024-09-21T03:10:16.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:16 smithi018 ceph-mon[34587]: pgmap v625: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 56 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 204 B/s wr, 3 op/s 2024-09-21T03:10:16.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:16 smithi123 ceph-mon[34645]: pgmap v625: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 56 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 204 B/s wr, 3 op/s 2024-09-21T03:10:17.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:16 smithi016 ceph-mon[31253]: pgmap v625: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 56 GiB data, 173 GiB used, 900 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 204 B/s wr, 3 op/s 2024-09-21T03:10:18.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:18 smithi018 ceph-mon[34587]: pgmap v626: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 55 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-21T03:10:18.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:18 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:10:18.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:18 smithi123 ceph-mon[34645]: pgmap v626: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 55 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-21T03:10:18.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:18 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:10:19.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:18 smithi016 ceph-mon[31253]: pgmap v626: 97 pgs: 22 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 37 active+clean; 55 GiB data, 171 GiB used, 902 GiB / 1.0 TiB avail; 4.3 KiB/s rd, 716 B/s wr, 5 op/s 2024-09-21T03:10:19.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:18 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:10:20.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:20 smithi018 ceph-mon[34587]: pgmap v627: 97 pgs: 19 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 44 active+clean; 55 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 655 B/s wr, 5 op/s 2024-09-21T03:10:20.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:20 smithi123 ceph-mon[34645]: pgmap v627: 97 pgs: 19 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 44 active+clean; 55 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 655 B/s wr, 5 op/s 2024-09-21T03:10:21.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:20 smithi016 ceph-mon[31253]: pgmap v627: 97 pgs: 19 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 44 active+clean; 55 GiB data, 168 GiB used, 905 GiB / 1.0 TiB avail; 5.1 KiB/s rd, 655 B/s wr, 5 op/s 2024-09-21T03:10:22.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:22 smithi018 ceph-mon[34587]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 53 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T03:10:22.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:22 smithi123 ceph-mon[34645]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 53 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T03:10:23.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:22 smithi016 ceph-mon[31253]: pgmap v628: 97 pgs: 13 active+clean+snaptrim, 28 active+clean+snaptrim_wait, 56 active+clean; 53 GiB data, 163 GiB used, 910 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 5 op/s 2024-09-21T03:10:24.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:24 smithi018 ceph-mon[34587]: pgmap v629: 97 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 62 active+clean; 52 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T03:10:24.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:24 smithi123 ceph-mon[34645]: pgmap v629: 97 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 62 active+clean; 52 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T03:10:25.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:24 smithi016 ceph-mon[31253]: pgmap v629: 97 pgs: 11 active+clean+snaptrim, 24 active+clean+snaptrim_wait, 62 active+clean; 52 GiB data, 161 GiB used, 911 GiB / 1.0 TiB avail; 4.2 KiB/s rd, 511 B/s wr, 4 op/s 2024-09-21T03:10:26.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:26 smithi123 ceph-mon[34645]: pgmap v630: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-21T03:10:27.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:26 smithi016 ceph-mon[31253]: pgmap v630: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-21T03:10:27.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:26 smithi018 ceph-mon[34587]: pgmap v630: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 4.6 KiB/s rd, 511 B/s wr, 5 op/s 2024-09-21T03:10:28.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:28 smithi123 ceph-mon[34645]: pgmap v631: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:10:29.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:28 smithi016 ceph-mon[31253]: pgmap v631: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:10:29.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:28 smithi018 ceph-mon[34587]: pgmap v631: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 51 GiB data, 157 GiB used, 916 GiB / 1.0 TiB avail; 2.8 KiB/s rd, 511 B/s wr, 3 op/s 2024-09-21T03:10:30.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:30 smithi123 ceph-mon[34645]: pgmap v632: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-21T03:10:31.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:30 smithi016 ceph-mon[31253]: pgmap v632: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-21T03:10:31.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:30 smithi018 ceph-mon[34587]: pgmap v632: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.9 KiB/s rd, 2 op/s 2024-09-21T03:10:32.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:32 smithi016 ceph-mon[31253]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:10:32.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:32 smithi016 ceph-mon[31253]: pgmap v634: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:32 smithi018 ceph-mon[34587]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:10:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:32 smithi018 ceph-mon[34587]: pgmap v634: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:32 smithi123 ceph-mon[34645]: osdmap e87: 12 total, 12 up, 12 in 2024-09-21T03:10:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:32 smithi123 ceph-mon[34645]: pgmap v634: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:34.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:34 smithi018 ceph-mon[34587]: pgmap v635: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:34.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:34 smithi123 ceph-mon[34645]: pgmap v635: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:35.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:34 smithi016 ceph-mon[31253]: pgmap v635: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:36 smithi018 ceph-mon[34587]: pgmap v636: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:36 smithi123 ceph-mon[34645]: pgmap v636: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:37.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:36 smithi016 ceph-mon[31253]: pgmap v636: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:38 smithi018 ceph-mon[34587]: pgmap v637: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:38 smithi123 ceph-mon[34645]: pgmap v637: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:38 smithi016 ceph-mon[31253]: pgmap v637: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:40.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:40 smithi018 ceph-mon[34587]: pgmap v638: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:40.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:40 smithi123 ceph-mon[34645]: pgmap v638: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:41.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:40 smithi016 ceph-mon[31253]: pgmap v638: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1023 B/s rd, 1 op/s 2024-09-21T03:10:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:42 smithi018 ceph-mon[34587]: pgmap v639: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-09-21T03:10:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:42 smithi123 ceph-mon[34645]: pgmap v639: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-09-21T03:10:43.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:42 smithi016 ceph-mon[31253]: pgmap v639: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 992 B/s rd, 1 op/s 2024-09-21T03:10:43.924 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:43 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:43.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:43 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:43.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:43 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:10:44.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:44 smithi018 ceph-mon[34587]: pgmap v640: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:44.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:44 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:44 smithi123 ceph-mon[34645]: pgmap v640: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:44.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:44 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:44 smithi016 ceph-mon[31253]: pgmap v640: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:45.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:44 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:45.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:45.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:10:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:10:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:10:46.289 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:10:46.561 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 -- 172.21.15.16:0/3458330430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af01094c0 msgr2=0x7f5af01098d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/3458330430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af01094c0 0x7f5af01098d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae002a410 comp rx=0 tx=0).stop 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 -- 172.21.15.16:0/3458330430 shutdown_connections 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/3458330430 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af01094c0 0x7f5af01098d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/3458330430 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 0x7f5af0110640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/3458330430 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af0103b80 0x7f5af0103f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 -- 172.21.15.16:0/3458330430 >> 172.21.15.16:0/3458330430 conn(0x7f5af00ff610 msgr2=0x7f5af0101a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 -- 172.21.15.16:0/3458330430 shutdown_connections 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.124+0000 7f5af5f2c640 1 -- 172.21.15.16:0/3458330430 wait complete. 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.125+0000 7f5af5f2c640 1 Processor -- start 2024-09-21T03:10:47.127 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.125+0000 7f5af5f2c640 1 -- start start 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af01094c0 0x7f5af019ab00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 0x7f5af019fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113ad0 con 0x7f5af0103b80 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113950 con 0x7f5af0109e10 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5af5f2c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af0113c50 con 0x7f5af01094c0 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33600/0 (socket says 172.21.15.16:33600) 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 -- 172.21.15.16:0/656758839 learned_addr learned my addr 172.21.15.16:0/656758839 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:47.129 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aeffff640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 0x7f5af019fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af01094c0 0x7f5af019ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 -- 172.21.15.16:0/656758839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af01094c0 msgr2=0x7f5af019ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af01094c0 0x7f5af019ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 -- 172.21.15.16:0/656758839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 msgr2=0x7f5af019fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 0x7f5af019fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.126+0000 7f5aef7fe640 1 -- 172.21.15.16:0/656758839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5af01a04d0 con 0x7f5af0103b80 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d770 tx=0x7f5ad800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5aecff9640 1 -- 172.21.15.16:0/656758839 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8017070 con 0x7f5af0103b80 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5aecff9640 1 -- 172.21.15.16:0/656758839 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ad80054d0 con 0x7f5af0103b80 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5af01a07c0 con 0x7f5af0103b80 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5aecff9640 1 -- 172.21.15.16:0/656758839 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8004c30 con 0x7f5af0103b80 2024-09-21T03:10:47.130 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.127+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5af01a6f20 con 0x7f5af0103b80 2024-09-21T03:10:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.129+0000 7f5afc4a0740 1 Processor -- start 2024-09-21T03:10:47.131 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.129+0000 7f5afc4a0740 1 -- start start 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 0x55577c6cb7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55577c641e50 con 0x55577c324070 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55577c631280 con 0x55577c2fb1b0 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55577c635090 con 0x55577c303390 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aecff9640 1 -- 172.21.15.16:0/656758839 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5ad8004e10 con 0x7f5af0103b80 2024-09-21T03:10:47.133 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33606/0 (socket says 172.21.15.16:33606) 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 -- 172.21.15.16:0/2996273021 learned_addr learned my addr 172.21.15.16:0/2996273021 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 0x55577c6cb7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 -- 172.21.15.16:0/2996273021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 msgr2=0x55577c6cbcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6cbcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 -- 172.21.15.16:0/2996273021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 msgr2=0x55577c6cb7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 0x55577c6cb7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aeffff640 1 -- 172.21.15.16:0/2996273021 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577c38f650 con 0x55577c324070 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.130+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 0x55577c6cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009f90 tx=0x7f5ae8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a3936b13bc2f1b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5ae5ffb640 1 -- 172.21.15.16:0/2996273021 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8011700 con 0x55577c324070 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5ae5ffb640 1 -- 172.21.15.16:0/2996273021 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8011d50 con 0x55577c324070 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5ae5ffb640 1 -- 172.21.15.16:0/2996273021 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8019da0 con 0x55577c324070 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5aecff9640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 0x7f5adc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.134 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.131+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 0x7f5adc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5aecff9640 1 -- 172.21.15.16:0/656758839 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f5ad80e71f0 con 0x7f5af0103b80 2024-09-21T03:10:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2996273021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 msgr2=0x55577c6cb270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.135 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009f90 tx=0x7f5ae8010860 comp rx=0 tx=0).stop 2024-09-21T03:10:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2996273021 shutdown_connections 2024-09-21T03:10:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6cbcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.137 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5aef7fe640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 0x7f5adc0b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55577c6c9ef0 tx=0x7f5ad80e9040 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c2fb1b0 0x55577c6cb7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2996273021 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6cb270 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2996273021 >> 172.21.15.16:0/2996273021 conn(0x55577c318460 msgr2=0x55577c6c9160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2996273021 shutdown_connections 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.132+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2996273021 wait complete. 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 -- start start 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 Processor -- start 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6d6990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6e2d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55577c492700 con 0x55577c303390 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55577c5cc110 con 0x55577c324070 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55577c6d9a50 con 0x55577c2fb1b0 2024-09-21T03:10:47.138 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5afc4a0740 10 client.0 _ll_register_callbacks cb 0x55577c5e5990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5aeeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33610/0 (socket says 172.21.15.16:33610) 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5aeeffd640 1 -- 172.21.15.16:0/799181710 learned_addr learned my addr 172.21.15.16:0/799181710 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.133+0000 7f5aeffff640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6d6990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeffff640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6e2d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 -- 172.21.15.16:0/799181710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 msgr2=0x55577c6d6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6d6990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 -- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 msgr2=0x55577c6e2d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6e2d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 -- 172.21.15.16:0/799181710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577c5f9c40 con 0x55577c303390 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae0032f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9eba9ced606cebc server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae0035040 con 0x55577c303390 2024-09-21T03:10:47.139 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f5ae0031860 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae8011d50 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5ae0004040 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5afc4a0740 10 client.16545 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.134+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55577c6e9b10 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8011d50 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5ae0004870 con 0x55577c303390 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 10 client.16545 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 10 client.16545 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 20 client.16545 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2e29f0 0x55577c6ed030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 4 client.16545 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55577c643da0 con 0x55577c2e29f0 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.135+0000 7f5aeffff640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2e29f0 0x55577c6ed030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.140 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.136+0000 7f5aceffd640 10 client.16545 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:47.141 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.136+0000 7f5aeffff640 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2e29f0 0x55577c6ed030 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b02888eda9d662a server_cookie=ecd572032f0e42ef in_seq=0 out_seq=0 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5aceffd640 1 -- 172.21.15.16:0/799181710 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55577c643da0 con 0x55577c2e29f0 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5aceffd640 10 client.16545 handle_command_reply: tid=0 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 client.16545 shutdown 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 msgr2=0x55577c6d6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7f5ae0009900 tx=0x7f5ae0032f60 comp rx=0 tx=0).stop 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 shutdown_connections 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2e29f0 0x55577c6ed030 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c303390 0x55577c6d6ed0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6e2d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/799181710 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6d6990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 >> 172.21.15.16:0/799181710 conn(0x55577c318460 msgr2=0x55577c6ce450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.142 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5aceffd640 0 client.16545 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 shutdown_connections 2024-09-21T03:10:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 1 -- 172.21.15.16:0/799181710 wait complete. 2024-09-21T03:10:47.143 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.137+0000 7f5afc4a0740 20 client.16545 trim_cache size 0 max 16384 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 Processor -- start 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 -- start start 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6e2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6ce370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ab5b0 con 0x55577c2fb1b0 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ade10 con 0x55577c324070 2024-09-21T03:10:47.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01abc00 con 0x55577c303390 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33622/0 (socket says 172.21.15.16:33622) 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5aeffff640 1 -- 172.21.15.16:0/2025712042 learned_addr learned my addr 172.21.15.16:0/2025712042 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.155+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6e2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6ce370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 -- 172.21.15.16:0/2025712042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 msgr2=0x55577c6e2c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6e2c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 -- 172.21.15.16:0/2025712042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 msgr2=0x55577c6ce370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6ce370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 -- 172.21.15.16:0/2025712042 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55577c6d58f0 con 0x55577c2fb1b0 2024-09-21T03:10:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5aeffff640 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80196b0 tx=0x7f5ae8010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=291075a74af724c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5ae57fa640 1 -- 172.21.15.16:0/2025712042 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae8019a30 con 0x55577c2fb1b0 2024-09-21T03:10:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5ae57fa640 1 -- 172.21.15.16:0/2025712042 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8020040 con 0x55577c2fb1b0 2024-09-21T03:10:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.156+0000 7f5ae57fa640 1 -- 172.21.15.16:0/2025712042 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae80117f0 con 0x55577c2fb1b0 2024-09-21T03:10:47.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2025712042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 msgr2=0x55577c6d6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f5ae80196b0 tx=0x7f5ae8010860 comp rx=0 tx=0).stop 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2025712042 shutdown_connections 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c324070 0x55577c6ce370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c303390 0x55577c6e2c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/2025712042 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c2fb1b0 0x55577c6d6e50 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2025712042 >> 172.21.15.16:0/2025712042 conn(0x55577c318460 msgr2=0x55577c6d6740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2025712042 shutdown_connections 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.157+0000 7f5afc4a0740 1 -- 172.21.15.16:0/2025712042 wait complete. 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 -- start start 2024-09-21T03:10:47.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 Processor -- start 2024-09-21T03:10:47.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6df6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c303390 0x55577c6dac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ac2d0 con 0x55577c324070 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ad740 con 0x55577c303390 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5af01ac9a0 con 0x55577c2fb1b0 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5afc4a0740 10 client.0 _ll_register_callbacks cb 0x7f5ae0005db0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5aeffff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33636/0 (socket says 172.21.15.16:33636) 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.158+0000 7f5aeffff640 1 -- 172.21.15.16:0/3513576921 learned_addr learned my addr 172.21.15.16:0/3513576921 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6df6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c303390 0x55577c6dac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 -- 172.21.15.16:0/3513576921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 msgr2=0x55577c6df6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6df6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 -- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c303390 msgr2=0x55577c6dac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c303390 0x55577c6dac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 -- 172.21.15.16:0/3513576921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55577c7569e0 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aeffff640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009a80 tx=0x7f5ae8019640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aff45b616768105c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aceffd640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ae801f040 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f5ae8003b70 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.159+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ae8020040 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5ae8011500 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 10 client.16551 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55577c757b70 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ae8020040 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5ae80053a0 con 0x55577c324070 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 10 client.16551 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 10 client.16551 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:47.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 20 client.16551 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2b7da0 0x55577c75a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 4 client.16551 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55577c63f850 con 0x55577c2b7da0 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.160+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2b7da0 0x55577c75a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5aceffd640 10 client.16551 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5aeeffd640 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2b7da0 0x55577c75a0b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6fbf5dcf3a6ba0 server_cookie=1092cdb4e9b367f9 in_seq=0 out_seq=0 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5aceffd640 1 -- 172.21.15.16:0/3513576921 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5202 (crc 0 0 0) 0x55577c63f850 con 0x55577c2b7da0 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5aceffd640 10 client.16551 handle_command_reply: tid=0 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 client.16551 shutdown 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 msgr2=0x55577c6db360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f5ae8009a80 tx=0x7f5ae8019640 comp rx=0 tx=0).stop 2024-09-21T03:10:47.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 shutdown_connections 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55577c2b7da0 0x55577c75a0b0 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55577c324070 0x55577c6db360 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55577c303390 0x55577c6dac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5aceffd640 0 client.16551 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 --2- 172.21.15.16:0/3513576921 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55577c2fb1b0 0x55577c6df6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.161+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 >> 172.21.15.16:0/3513576921 conn(0x55577c318460 msgr2=0x55577c6ce370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 shutdown_connections 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5afc4a0740 1 -- 172.21.15.16:0/3513576921 wait complete. 2024-09-21T03:10:47.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5afc4a0740 20 client.16551 trim_cache size 0 max 16384 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "749c373e-c09d-4335-b4aa-d8d285c516ab": { 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:13.847478+0000", 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.167 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:10:47.168 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:10:47.169 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.170 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.172 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.173 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.174 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 msgr2=0x7f5adc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 0x7f5adc0b8b90 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x55577c6c9ef0 tx=0x7f5ad80e9040 comp rx=0 tx=0).stop 2024-09-21T03:10:47.176 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 msgr2=0x7f5af019a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800d770 tx=0x7f5ad800dc30 comp rx=0 tx=0).stop 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 shutdown_connections 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f5adc0b6680 0x7f5adc0b8b90 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5af0109e10 0x7f5af019fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5af01094c0 0x7f5af019ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 --2- 172.21.15.16:0/656758839 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5af0103b80 0x7f5af019a560 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 >> 172.21.15.16:0/656758839 conn(0x7f5af00ff610 msgr2=0x7f5af0101a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.162+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 shutdown_connections 2024-09-21T03:10:47.177 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:47.163+0000 7f5af5f2c640 1 -- 172.21.15.16:0/656758839 wait complete. 2024-09-21T03:10:47.305 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:47 smithi016 ceph-mon[31253]: pgmap v641: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:47.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:47 smithi018 ceph-mon[34587]: pgmap v641: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:47.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:47 smithi123 ceph-mon[34645]: pgmap v641: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:48.006 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:10:48.007 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:10:48.270 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:48.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 -- 172.21.15.16:0/2317196961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0102a60 msgr2=0x7f9ab0102e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 --2- 172.21.15.16:0/2317196961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0102a60 0x7f9ab0102e70 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f9aa0009ca0 tx=0x7f9aa002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 -- 172.21.15.16:0/2317196961 shutdown_connections 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 --2- 172.21.15.16:0/2317196961 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab0108cf0 0x7f9ab010f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 --2- 172.21.15.16:0/2317196961 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab01083a0 0x7f9ab01087b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 --2- 172.21.15.16:0/2317196961 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0102a60 0x7f9ab0102e70 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 -- 172.21.15.16:0/2317196961 >> 172.21.15.16:0/2317196961 conn(0x7f9ab0079730 msgr2=0x7f9ab0079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 -- 172.21.15.16:0/2317196961 shutdown_connections 2024-09-21T03:10:48.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 -- 172.21.15.16:0/2317196961 wait complete. 2024-09-21T03:10:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.781+0000 7f9ab731b640 1 Processor -- start 2024-09-21T03:10:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 -- start start 2024-09-21T03:10:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 0x7f9ab0104250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab01083a0 0x7f9ab0104790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:48.785 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0112960 con 0x7f9ab0108cf0 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab01127e0 con 0x7f9ab0102a60 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab731b640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab0112ae0 con 0x7f9ab01083a0 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33652/0 (socket says 172.21.15.16:33652) 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 -- 172.21.15.16:0/1706274972 learned_addr learned my addr 172.21.15.16:0/1706274972 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab48b6640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab01083a0 0x7f9ab0104790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab50b7640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 0x7f9ab0104250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 -- 172.21.15.16:0/1706274972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab01083a0 msgr2=0x7f9ab0104790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab01083a0 0x7f9ab0104790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 -- 172.21.15.16:0/1706274972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 msgr2=0x7f9ab0104250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:48.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 0x7f9ab0104250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab58b8640 1 -- 172.21.15.16:0/1706274972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab01a86e0 con 0x7f9ab0108cf0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.782+0000 7f9ab50b7640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 0x7f9ab0104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9ab58b8640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800e970 tx=0x7f9aa800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa800ccc0 con 0x7f9ab0108cf0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab01a89d0 con 0x7f9ab0108cf0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa800ce60 con 0x7f9ab0108cf0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8010430 con 0x7f9ab0108cf0 2024-09-21T03:10:48.787 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.783+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab01a8f10 con 0x7f9ab0108cf0 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.784+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9ab0102e70 con 0x7f9ab0108cf0 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.786+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f9aa8004fb0 con 0x7f9ab0108cf0 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.786+0000 7f9aae7fc640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 0x7f9a980b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.786+0000 7f9ab50b7640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 0x7f9a980b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.786+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f9aa80e7d90 con 0x7f9ab0108cf0 2024-09-21T03:10:48.788 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.787+0000 7f9ab50b7640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 0x7f9a980b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00094e0 tx=0x7f9aa00093f0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:48.791 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:48.790+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f9aa8056710 con 0x7f9ab0108cf0 2024-09-21T03:10:49.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.018+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9ab01a92e0 con 0x7f9ab0108cf0 2024-09-21T03:10:49.020 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.019+0000 7f9aae7fc640 1 -- 172.21.15.16:0/1706274972 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f9aa80ae2d0 con 0x7f9ab0108cf0 2024-09-21T03:10:49.020 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:10:49.021 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:10:49.021 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 msgr2=0x7f9a980b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 0x7f9a980b8b90 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f9aa00094e0 tx=0x7f9aa00093f0 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 msgr2=0x7f9ab0104cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800e970 tx=0x7f9aa800ed70 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 shutdown_connections 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f9a980b6680 0x7f9a980b8b90 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f9ab0108cf0 0x7f9ab0104cd0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ab01083a0 0x7f9ab0104790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 --2- 172.21.15.16:0/1706274972 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f9ab0102a60 0x7f9ab0104250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 >> 172.21.15.16:0/1706274972 conn(0x7f9ab0079730 msgr2=0x7f9ab0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 shutdown_connections 2024-09-21T03:10:49.023 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:49.021+0000 7f9ab731b640 1 -- 172.21.15.16:0/1706274972 wait complete. 2024-09-21T03:10:49.333 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:49 smithi016 ceph-mon[31253]: pgmap v642: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:49.334 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:49 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1706274972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:49.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:49 smithi018 ceph-mon[34587]: pgmap v642: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:49.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:49 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1706274972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:49.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:49 smithi123 ceph-mon[34645]: pgmap v642: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:49.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:49 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1706274972' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:10:49.631 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:10:49.892 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 -- 172.21.15.16:0/1230137141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 msgr2=0x7fde88110300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/1230137141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88110300 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fde80009900 tx=0x7fde8002a410 comp rx=0 tx=0).stop 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 -- 172.21.15.16:0/1230137141 shutdown_connections 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/1230137141 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88110300 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/1230137141 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 0x7fde8810d850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.424 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/1230137141 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 0x7fde88104dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 -- 172.21.15.16:0/1230137141 >> 172.21.15.16:0/1230137141 conn(0x7fde88079730 msgr2=0x7fde88079b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 -- 172.21.15.16:0/1230137141 shutdown_connections 2024-09-21T03:10:50.425 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.422+0000 7fde8cb7c640 1 -- 172.21.15.16:0/1230137141 wait complete. 2024-09-21T03:10:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.423+0000 7fde8cb7c640 1 Processor -- start 2024-09-21T03:10:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.423+0000 7fde8cb7c640 1 -- start start 2024-09-21T03:10:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 0x7fde881026f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.426 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 0x7fde88102c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde88113a70 con 0x7fde8810de10 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde881138f0 con 0x7fde88105300 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8cb7c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde88113bf0 con 0x7fde881049b0 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde8659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 0x7fde881026f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33666/0 (socket says 172.21.15.16:33666) 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 -- 172.21.15.16:0/448465460 learned_addr learned my addr 172.21.15.16:0/448465460 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde85d9b640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 0x7fde88102c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 -- 172.21.15.16:0/448465460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 msgr2=0x7fde881026f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 0x7fde881026f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 -- 172.21.15.16:0/448465460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 msgr2=0x7fde88102c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 0x7fde88102c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.427 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 -- 172.21.15.16:0/448465460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde881a85d0 con 0x7fde8810de10 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.424+0000 7fde86d9d640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fde80009a20 tx=0x7fde80004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.425+0000 7fde777fe640 1 -- 172.21.15.16:0/448465460 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde8003f030 con 0x7fde8810de10 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.425+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a8860 con 0x7fde8810de10 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.425+0000 7fde777fe640 1 -- 172.21.15.16:0/448465460 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde80003aa0 con 0x7fde8810de10 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.425+0000 7fde777fe640 1 -- 172.21.15.16:0/448465460 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80035070 con 0x7fde8810de10 2024-09-21T03:10:50.428 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.425+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881a8d90 con 0x7fde8810de10 2024-09-21T03:10:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.427+0000 7fde930f0740 1 Processor -- start 2024-09-21T03:10:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.427+0000 7fde777fe640 1 -- 172.21.15.16:0/448465460 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fde80042030 con 0x7fde8810de10 2024-09-21T03:10:50.429 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 -- start start 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 0x558434dac2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349dc1b0 0x558434dac800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 0x558434dacd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558434d22ea0 con 0x5584349e4390 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558434d122d0 con 0x558434a05070 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde930f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558434d160e0 con 0x5584349dc1b0 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 0x558434dacd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 0x558434dacd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33678/0 (socket says 172.21.15.16:33678) 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.428+0000 7fde85d9b640 1 -- 172.21.15.16:0/1772507415 learned_addr learned my addr 172.21.15.16:0/1772507415 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 0x558434dac2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.431 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde8659c640 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349dc1b0 0x558434dac800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde777fe640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 0x7fde700b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 -- 172.21.15.16:0/1772507415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349dc1b0 msgr2=0x558434dac800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349dc1b0 0x558434dac800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 -- 172.21.15.16:0/1772507415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 msgr2=0x558434dacd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 0x558434dacd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 -- 172.21.15.16:0/1772507415 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558434a70650 con 0x558434a05070 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde86d9d640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 0x7fde700b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde777fe640 1 -- 172.21.15.16:0/448465460 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fde801078a0 con 0x7fde8810de10 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde85d9b640 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 0x558434dac2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005c40 tx=0x7fde7c0075b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=969d34a5fb67e58 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde74ff9640 1 -- 172.21.15.16:0/1772507415 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c011e80 con 0x558434a05070 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde74ff9640 1 -- 172.21.15.16:0/1772507415 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c014750 con 0x558434a05070 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde74ff9640 1 -- 172.21.15.16:0/1772507415 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005890 con 0x558434a05070 2024-09-21T03:10:50.432 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.429+0000 7fde86d9d640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 0x7fde700b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde80105090 tx=0x7fde8009a440 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 -- 172.21.15.16:0/1772507415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 msgr2=0x558434dac2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 0x558434dac2c0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fde7c005c40 tx=0x7fde7c0075b0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 -- 172.21.15.16:0/1772507415 shutdown_connections 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349e4390 0x558434dacd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349dc1b0 0x558434dac800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 --2- 172.21.15.16:0/1772507415 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x558434a05070 0x558434dac2c0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.430+0000 7fde930f0740 1 -- 172.21.15.16:0/1772507415 >> 172.21.15.16:0/1772507415 conn(0x5584349f9460 msgr2=0x558434daa1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- 172.21.15.16:0/1772507415 shutdown_connections 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- 172.21.15.16:0/1772507415 wait complete. 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- start start 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 Processor -- start 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349dc1b0 0x558434db7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349e4390 0x558434db7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x558434b73700 con 0x558434a05070 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x558434cad160 con 0x5584349dc1b0 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x558434dbaaa0 con 0x5584349e4390 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde930f0740 10 client.0 _ll_register_callbacks cb 0x558434cc69e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.431+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33690/0 (socket says 172.21.15.16:33690) 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 -- 172.21.15.16:0/2007622429 learned_addr learned my addr 172.21.15.16:0/2007622429 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349dc1b0 0x558434db7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde8659c640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349e4390 0x558434db7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 -- 172.21.15.16:0/2007622429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349e4390 msgr2=0x558434db7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349e4390 0x558434db7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 -- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349dc1b0 msgr2=0x558434db7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349dc1b0 0x558434db7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 -- 172.21.15.16:0/2007622429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558434cdac90 con 0x558434a05070 2024-09-21T03:10:50.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fde7c014a80 tx=0x7fde7c005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d130df747a81fee server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005820 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7c014750 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fde7c004a40 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fde7c004c40 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde930f0740 10 client.16569 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.432+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558434dc67b0 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c014750 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fde7c00f440 con 0x558434a05070 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 10 client.16569 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 10 client.16569 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 20 client.16569 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5584349c39f0 0x558434dc9bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 4 client.16569 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:50.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558434d24df0 con 0x5584349c39f0 2024-09-21T03:10:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.433+0000 7fde8659c640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5584349c39f0 0x558434dc9bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.434+0000 7fde6dffb640 10 client.16569 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:50.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.434+0000 7fde8659c640 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5584349c39f0 0x558434dc9bc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3e69c7892bafcd91 server_cookie=d7eab35e83a4a2c8 in_seq=0 out_seq=0 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.434+0000 7fde6dffb640 1 -- 172.21.15.16:0/2007622429 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x558434d24df0 con 0x5584349c39f0 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.434+0000 7fde6dffb640 10 client.16569 handle_command_reply: tid=0 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 client.16569 shutdown 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 msgr2=0x558434dc3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fde7c014a80 tx=0x7fde7c005470 comp rx=0 tx=0).stop 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 shutdown_connections 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x5584349c39f0 0x558434dc9bc0 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x558434a05070 0x558434dc3dc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5584349e4390 0x558434db7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 --2- 172.21.15.16:0/2007622429 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349dc1b0 0x558434db7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde6dffb640 0 client.16569 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 >> 172.21.15.16:0/2007622429 conn(0x5584349f9460 msgr2=0x558434daf450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 shutdown_connections 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 1 -- 172.21.15.16:0/2007622429 wait complete. 2024-09-21T03:10:50.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.435+0000 7fde930f0740 20 client.16569 trim_cache size 0 max 16384 2024-09-21T03:10:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 Processor -- start 2024-09-21T03:10:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 -- start start 2024-09-21T03:10:50.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dc3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434db4b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde881af840 con 0x5584349dc1b0 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b20a0 con 0x5584349e4390 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde930f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde881afe90 con 0x558434a05070 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33700/0 (socket says 172.21.15.16:33700) 2024-09-21T03:10:50.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 -- 172.21.15.16:0/2145039115 learned_addr learned my addr 172.21.15.16:0/2145039115 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde8659c640 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dc3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 -- 172.21.15.16:0/2145039115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 msgr2=0x558434db4b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434db4b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 -- 172.21.15.16:0/2145039115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 msgr2=0x558434dc3cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dc3cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde85d9b640 1 -- 172.21.15.16:0/2145039115 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558434daaaa0 con 0x5584349dc1b0 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.449+0000 7fde8659c640 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dc3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde85d9b640 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fde7c011c40 tx=0x7fde7c00ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1d7090831cb1c0b server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde6ffff640 1 -- 172.21.15.16:0/2145039115 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005e90 con 0x5584349dc1b0 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde6ffff640 1 -- 172.21.15.16:0/2145039115 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c025070 con 0x5584349dc1b0 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde6ffff640 1 -- 172.21.15.16:0/2145039115 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c005820 con 0x5584349dc1b0 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 -- 172.21.15.16:0/2145039115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 msgr2=0x558434db9580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fde7c011c40 tx=0x7fde7c00ff40 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 -- 172.21.15.16:0/2145039115 shutdown_connections 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434db4b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dc3cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 --2- 172.21.15.16:0/2145039115 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434db9580 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.453 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.450+0000 7fde930f0740 1 -- 172.21.15.16:0/2145039115 >> 172.21.15.16:0/2145039115 conn(0x5584349f9460 msgr2=0x558434db8e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- 172.21.15.16:0/2145039115 shutdown_connections 2024-09-21T03:10:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- 172.21.15.16:0/2145039115 wait complete. 2024-09-21T03:10:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- start start 2024-09-21T03:10:50.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 Processor -- start 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dbb9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434dbbef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b0560 con 0x5584349dc1b0 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b19d0 con 0x5584349e4390 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b0c30 con 0x558434a05070 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde930f0740 10 client.0 _ll_register_callbacks cb 0x7fde7c007d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.451+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33706/0 (socket says 172.21.15.16:33706) 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 -- 172.21.15.16:0/213273669 learned_addr learned my addr 172.21.15.16:0/213273669 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde8659c640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dbb9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.457 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 -- 172.21.15.16:0/213273669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 msgr2=0x558434dbbef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434dbbef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 -- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 msgr2=0x558434dbb9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dbb9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 -- 172.21.15.16:0/213273669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558434dbc680 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde85d9b640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fde7c008010 tx=0x7fde7c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d548b9de5e260115 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde6dffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c0075b0 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fde7c005820 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde7c025070 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fde7c00fa80 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde930f0740 10 client.16575 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.452+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x558434e34280 con 0x5584349dc1b0 2024-09-21T03:10:50.458 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c025070 con 0x5584349dc1b0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fde7c012a90 con 0x5584349dc1b0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 10 client.16575 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 10 client.16575 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 20 client.16575 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558434998da0 0x558434e368d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 4 client.16575 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558434d208a0 con 0x558434998da0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde8659c640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558434998da0 0x558434e368d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.453+0000 7fde8659c640 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558434998da0 0x558434e368d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2c1e45cebe7989be server_cookie=e27f45b319787935 in_seq=0 out_seq=0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde6dffb640 10 client.16575 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde6dffb640 1 -- 172.21.15.16:0/213273669 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558434d208a0 con 0x558434998da0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde6dffb640 10 client.16575 handle_command_reply: tid=0 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 client.16575 shutdown 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 msgr2=0x558434dc0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.459 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fde7c008010 tx=0x7fde7c005fd0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 shutdown_connections 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x558434998da0 0x558434e368d0 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x558434a05070 0x558434dbbef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x5584349e4390 0x558434dbb9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 --2- 172.21.15.16:0/213273669 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x5584349dc1b0 0x558434dc0500 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde6dffb640 0 client.16575 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 >> 172.21.15.16:0/213273669 conn(0x5584349f9460 msgr2=0x558434db8d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 shutdown_connections 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 1 -- 172.21.15.16:0/213273669 wait complete. 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.454+0000 7fde930f0740 20 client.16575 trim_cache size 0 max 16384 2024-09-21T03:10:50.460 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 msgr2=0x7fde700b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 0x7fde700b8c10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fde80105090 tx=0x7fde8009a440 comp rx=0 tx=0).stop 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 msgr2=0x7fde88103170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fde80009a20 tx=0x7fde80004270 comp rx=0 tx=0).stop 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 shutdown_connections 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fde700b6700 0x7fde700b8c10 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fde8810de10 0x7fde88103170 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.461 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fde88105300 0x7fde88102c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 --2- 172.21.15.16:0/448465460 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde881049b0 0x7fde881026f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 >> 172.21.15.16:0/448465460 conn(0x7fde88079730 msgr2=0x7fde880ff550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 shutdown_connections 2024-09-21T03:10:50.462 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:50.455+0000 7fde8cb7c640 1 -- 172.21.15.16:0/448465460 wait complete. 2024-09-21T03:10:51.169 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:51 smithi016 ceph-mon[31253]: pgmap v643: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:51.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:51 smithi018 ceph-mon[34587]: pgmap v643: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:51.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:51 smithi123 ceph-mon[34645]: pgmap v643: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 2 op/s 2024-09-21T03:10:52.199 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-09-21T03:10:52.199 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub start / force,recursive 2024-09-21T03:10:52.473 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:52.994 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 -- 172.21.15.16:0/874854748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 msgr2=0x7fc8f8102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/874854748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f8102e90 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8009900 tx=0x7fc8e802a410 comp rx=0 tx=0).stop 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 -- 172.21.15.16:0/874854748 shutdown_connections 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/874854748 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 0x7fc8f810f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/874854748 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f81087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/874854748 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f8102e90 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 -- 172.21.15.16:0/874854748 >> 172.21.15.16:0/874854748 conn(0x7fc8f8079740 msgr2=0x7fc8f8079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 -- 172.21.15.16:0/874854748 shutdown_connections 2024-09-21T03:10:52.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.993+0000 7fc8fcea7640 1 -- 172.21.15.16:0/874854748 wait complete. 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 Processor -- start 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 -- start start 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f8199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 0x7fc8f819ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8112980 con 0x7fc8f8102a80 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8112800 con 0x7fc8f8108d10 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8fcea7640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f8112b00 con 0x7fc8f81083c0 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:52.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33722/0 (socket says 172.21.15.16:33722) 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8f659c640 1 -- 172.21.15.16:0/483578873 learned_addr learned my addr 172.21.15.16:0/483578873 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f8199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.994+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 0x7fc8f819ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 -- 172.21.15.16:0/483578873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 msgr2=0x7fc8f8199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f8199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 -- 172.21.15.16:0/483578873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 msgr2=0x7fc8f819ec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 0x7fc8f819ec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 -- 172.21.15.16:0/483578873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc8f819f360 con 0x7fc8f8102a80 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8f659c640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005940 tx=0x7fc8e80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8e77fe640 1 -- 172.21.15.16:0/483578873 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e803f030 con 0x7fc8f8102a80 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc8f819f5f0 con 0x7fc8f8102a80 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8e77fe640 1 -- 172.21.15.16:0/483578873 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e8003cb0 con 0x7fc8f8102a80 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8e77fe640 1 -- 172.21.15.16:0/483578873 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8035070 con 0x7fc8f8102a80 2024-09-21T03:10:52.998 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.995+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8f806bb40 con 0x7fc8f8102a80 2024-09-21T03:10:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 Processor -- start 2024-09-21T03:10:52.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 -- start start 2024-09-21T03:10:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61e3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa61e3970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fa6159e60 con 0x562fa5e1b670 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562fa6149290 con 0x562fa5e3c350 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc90341b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562fa614d0a0 con 0x562fa5e13490 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc8e77fe640 1 -- 172.21.15.16:0/483578873 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fc8e8042030 con 0x7fc8f8102a80 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33738/0 (socket says 172.21.15.16:33738) 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc8f6d9d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61e3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.998+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/3724664267 learned_addr learned my addr 172.21.15.16:0/3724664267 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f659c640 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa61e3970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/3724664267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 msgr2=0x562fa61e3970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa61e3970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/3724664267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 msgr2=0x562fa61e3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61e3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/3724664267 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa5ea7930 con 0x562fa5e1b670 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f659c640 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa61e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:53.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00cc10 tx=0x7fc8ec00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1353fba7100d88c0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:53.004 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8e4ff9640 1 -- 172.21.15.16:0/3724664267 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec014070 con 0x562fa5e1b670 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8e4ff9640 1 -- 172.21.15.16:0/3724664267 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0047a0 con 0x562fa5e1b670 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:52.999+0000 7fc8e4ff9640 1 -- 172.21.15.16:0/3724664267 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec010630 con 0x562fa5e1b670 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc8e77fe640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 0x7fc8e00b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 0x7fc8e00b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc8e77fe640 1 -- 172.21.15.16:0/483578873 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fc8e8108190 con 0x7fc8f8102a80 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 -- 172.21.15.16:0/3724664267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 msgr2=0x562fa61e3eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec00cc10 tx=0x7fc8ec00e720 comp rx=0 tx=0).stop 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 0x7fc8e00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005fd0 tx=0x7fc8f0009760 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 -- 172.21.15.16:0/3724664267 shutdown_connections 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61e3eb0 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa61e3970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 --2- 172.21.15.16:0/3724664267 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61e3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 -- 172.21.15.16:0/3724664267 >> 172.21.15.16:0/3724664267 conn(0x562fa5e30740 msgr2=0x562fa61e1320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:53.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.000+0000 7fc90341b740 1 -- 172.21.15.16:0/3724664267 shutdown_connections 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 -- 172.21.15.16:0/3724664267 wait complete. 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 -- start start 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 Processor -- start 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e13490 0x562fa61ef030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e3c350 0x562fa61faf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x562fa5faa890 con 0x562fa5e1b670 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x562fa60e3d40 con 0x562fa5e13490 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x562fa61f1c10 con 0x562fa5e3c350 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.001+0000 7fc90341b740 10 client.0 _ll_register_callbacks cb 0x562fa6147c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33742/0 (socket says 172.21.15.16:33742) 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/527475696 learned_addr learned my addr 172.21.15.16:0/527475696 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e3c350 0x562fa61faf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.006 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f659c640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e13490 0x562fa61ef030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/527475696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e3c350 msgr2=0x562fa61faf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e3c350 0x562fa61faf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e13490 msgr2=0x562fa61ef030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e13490 0x562fa61ef030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 -- 172.21.15.16:0/527475696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa61fb540 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec010b70 tx=0x7fc8ec010ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a11a5df10fc7272 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8ec027060 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fc8ec022410 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.002+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc8ec0047a0 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc8ec014070 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 10 client.16587 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562fa61fde70 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8ec0047a0 con 0x562fa5e1b670 2024-09-21T03:10:53.007 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc8ec0184a0 con 0x562fa5e1b670 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 10 client.16587 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 10 client.16587 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 20 client.16587 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dfacd0 0x562fa62011d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 4 client.16587 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562fa615bdb0 con 0x562fa5dfacd0 2024-09-21T03:10:53.008 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.003+0000 7fc8f659c640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dfacd0 0x562fa62011d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.004+0000 7fc8f659c640 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dfacd0 0x562fa62011d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b26db4e7b75502a server_cookie=927461a7e39f9455 in_seq=0 out_seq=0 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.004+0000 7fc8ddffb640 10 client.16587 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.004+0000 7fc8ddffb640 1 -- 172.21.15.16:0/527475696 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x562fa615bdb0 con 0x562fa5dfacd0 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.004+0000 7fc8ddffb640 10 client.16587 handle_command_reply: tid=0 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 client.16587 shutdown 2024-09-21T03:10:53.009 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 msgr2=0x562fa61ef5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fc8ec010b70 tx=0x7fc8ec010ba0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 shutdown_connections 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dfacd0 0x562fa62011d0 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e1b670 0x562fa61ef5b0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e3c350 0x562fa61faf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 --2- 172.21.15.16:0/527475696 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e13490 0x562fa61ef030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 >> 172.21.15.16:0/527475696 conn(0x562fa5e30740 msgr2=0x562fa61e65c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc8ddffb640 0 client.16587 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 shutdown_connections 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 1 -- 172.21.15.16:0/527475696 wait complete. 2024-09-21T03:10:53.010 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.005+0000 7fc90341b740 20 client.16587 trim_cache size 0 max 16384 2024-09-21T03:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 Processor -- start 2024-09-21T03:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 -- start start 2024-09-21T03:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e1b670 0x562fa61ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.025 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61edbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81ad740 con 0x562fa5e13490 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81affa0 con 0x562fa5e3c350 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc90341b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81add90 con 0x562fa5e1b670 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33756/0 (socket says 172.21.15.16:33756) 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 -- 172.21.15.16:0/1535276337 learned_addr learned my addr 172.21.15.16:0/1535276337 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61edbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e1b670 0x562fa61ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 -- 172.21.15.16:0/1535276337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e1b670 msgr2=0x562fa61ed6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e1b670 0x562fa61ed6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 -- 172.21.15.16:0/1535276337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 msgr2=0x562fa61edbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61edbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.023+0000 7fc8f659c640 1 -- 172.21.15.16:0/1535276337 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa61efed0 con 0x562fa5e13490 2024-09-21T03:10:53.026 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.024+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80cdc70 tx=0x7fc8e80f1040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fedb400fdd33fc0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.024+0000 7fc8dffff640 1 -- 172.21.15.16:0/1535276337 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8049040 con 0x562fa5e13490 2024-09-21T03:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.024+0000 7fc8dffff640 1 -- 172.21.15.16:0/1535276337 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8e803b460 con 0x562fa5e13490 2024-09-21T03:10:53.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.024+0000 7fc8dffff640 1 -- 172.21.15.16:0/1535276337 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e80d7990 con 0x562fa5e13490 2024-09-21T03:10:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- 172.21.15.16:0/1535276337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 msgr2=0x562fa61faea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fc8e80cdc70 tx=0x7fc8e80f1040 comp rx=0 tx=0).stop 2024-09-21T03:10:53.029 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- 172.21.15.16:0/1535276337 shutdown_connections 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e3c350 0x562fa61edbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e1b670 0x562fa61ed6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- 172.21.15.16:0/1535276337 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e13490 0x562fa61faea0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- 172.21.15.16:0/1535276337 >> 172.21.15.16:0/1535276337 conn(0x562fa5e30740 msgr2=0x562fa61e9f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- 172.21.15.16:0/1535276337 shutdown_connections 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- 172.21.15.16:0/1535276337 wait complete. 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 -- start start 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 Processor -- start 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa6202280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.025+0000 7fc90341b740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 0x562fa62027c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc90341b740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc90341b740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81ae460 con 0x562fa5e3c350 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc90341b740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81af8d0 con 0x562fa5e1b670 2024-09-21T03:10:53.030 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc90341b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fc8f81aeb30 con 0x562fa5e13490 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc90341b740 10 client.0 _ll_register_callbacks cb 0x7fc8f0009600 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa6202280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f5d9b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 0x562fa62027c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f6d9d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:33772/0 (socket says 172.21.15.16:33772) 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f6d9d640 1 -- 172.21.15.16:0/1645154527 learned_addr learned my addr 172.21.15.16:0/1645154527 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 -- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 msgr2=0x562fa62027c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 0x562fa62027c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 -- 172.21.15.16:0/1645154527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 msgr2=0x562fa6202d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 -- 172.21.15.16:0/1645154527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa626a9d0 con 0x562fa5e13490 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f6d9d640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 0x562fa62027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8f659c640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa6202280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x562fa60e3e90 tx=0x7fc8e8009680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5788e5fddb44d5cf server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8ddffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:10:53.031 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.026+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8e8049070 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7fc8e80d7700 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x562fa6157860 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fc8e8105d00 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 10 client.26209 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x562fa626bc90 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x562fa6157860 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fc8e803bb50 con 0x562fa5e13490 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 10 client.26209 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 10 client.26209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 20 client.26209 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dd0080 0x562fa626e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 4 client.26209 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.027+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x562fa6154c90 con 0x562fa5dd0080 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.028+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dd0080 0x562fa626e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:53.032 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.028+0000 7fc8ddffb640 10 client.26209 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.028+0000 7fc8f5d9b640 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dd0080 0x562fa626e3f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7809e0d440e97e4d server_cookie=e687f5c236130eae in_seq=0 out_seq=0 2024-09-21T03:10:53.033 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc8ddffb640 1 -- 172.21.15.16:0/1645154527 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x562fa6154c90 con 0x562fa5dd0080 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc8ddffb640 10 client.26209 handle_command_reply: tid=0 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 client.26209 shutdown 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 msgr2=0x562fa6202280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa6202280 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x562fa60e3e90 tx=0x7fc8e8009680 comp rx=0 tx=0).stop 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 shutdown_connections 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x562fa5dd0080 0x562fa626e3f0 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc8ddffb640 0 client.26209 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x562fa5e3c350 0x562fa6202d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x562fa5e1b670 0x562fa62027c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 --2- 172.21.15.16:0/1645154527 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x562fa5e13490 0x562fa6202280 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 >> 172.21.15.16:0/1645154527 conn(0x562fa5e30740 msgr2=0x562fa61e8540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 shutdown_connections 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 1 -- 172.21.15.16:0/1645154527 wait complete. 2024-09-21T03:10:53.034 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.030+0000 7fc90341b740 20 client.26209 trim_cache size 0 max 16384 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stdout: "return_code": 0, 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_tag": "baf763ad-6228-4e74-a645-aeb506b4e304", 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": "asynchronous" 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:53.035 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 msgr2=0x7fc8e00b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 0x7fc8e00b8b90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fc8f0005fd0 tx=0x7fc8f0009760 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 msgr2=0x7fc8f81993f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fc8e8005940 tx=0x7fc8e80042f0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 shutdown_connections 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fc8e00b6680 0x7fc8e00b8b90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fc8f8108d10 0x7fc8f819ec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc8f81083c0 0x7fc8f8199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 --2- 172.21.15.16:0/483578873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fc8f8102a80 0x7fc8f81993f0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 >> 172.21.15.16:0/483578873 conn(0x7fc8f8079740 msgr2=0x7fc8f81014d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 shutdown_connections 2024-09-21T03:10:53.036 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:53.031+0000 7fc8fcea7640 1 -- 172.21.15.16:0/483578873 wait complete. 2024-09-21T03:10:53.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:53 smithi016 ceph-mon[31253]: pgmap v644: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:53.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:53 smithi018 ceph-mon[34587]: pgmap v644: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:53.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:53 smithi123 ceph-mon[34645]: pgmap v644: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 2 op/s 2024-09-21T03:10:53.706 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:10:53.965 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:10:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:54 smithi016 ceph-mon[31253]: scrub queued for path: / 2024-09-21T03:10:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:54 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:54 smithi016 ceph-mon[31253]: scrub summary: active paths [/] 2024-09-21T03:10:54.338 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:54 smithi016 ceph-mon[31253]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:54 smithi018 ceph-mon[34587]: scrub queued for path: / 2024-09-21T03:10:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:54 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:54 smithi018 ceph-mon[34587]: scrub summary: active paths [/] 2024-09-21T03:10:54.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:54 smithi018 ceph-mon[34587]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 -- 172.21.15.16:0/2535000935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56741049d0 msgr2=0x7f5674104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 --2- 172.21.15.16:0/2535000935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56741049d0 0x7f5674104de0 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7f5664009900 tx=0x7f566402a410 comp rx=0 tx=0).stop 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 -- 172.21.15.16:0/2535000935 shutdown_connections 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 --2- 172.21.15.16:0/2535000935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 0x7f5674110320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 --2- 172.21.15.16:0/2535000935 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5674105320 0x7f567410d870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 --2- 172.21.15.16:0/2535000935 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f56741049d0 0x7f5674104de0 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 -- 172.21.15.16:0/2535000935 >> 172.21.15.16:0/2535000935 conn(0x7f5674079740 msgr2=0x7f5674079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 -- 172.21.15.16:0/2535000935 shutdown_connections 2024-09-21T03:10:54.433 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.431+0000 7f567a28c640 1 -- 172.21.15.16:0/2535000935 wait complete. 2024-09-21T03:10:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.432+0000 7f567a28c640 1 Processor -- start 2024-09-21T03:10:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.432+0000 7f567a28c640 1 -- start start 2024-09-21T03:10:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56741049d0 0x7f567419a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 0x7f567419fd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f5674113ab0 con 0x7f5674105320 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5674113930 con 0x7f56741049d0 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f567a28c640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5674113c30 con 0x7f567410de30 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47074/0 (socket says 172.21.15.16:47074) 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 -- 172.21.15.16:0/2074787531 learned_addr learned my addr 172.21.15.16:0/2074787531 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f5678829640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 0x7f567419fd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f5673fff640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56741049d0 0x7f567419a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 -- 172.21.15.16:0/2074787531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 msgr2=0x7f567419fd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 0x7f567419fd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 -- 172.21.15.16:0/2074787531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56741049d0 msgr2=0x7f567419a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56741049d0 0x7f567419a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 -- 172.21.15.16:0/2074787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56741a0470 con 0x7f5674105320 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56737fe640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f566800d950 tx=0x7f566800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.433+0000 7f56717fa640 1 -- 172.21.15.16:0/2074787531 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5668017070 con 0x7f5674105320 2024-09-21T03:10:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.434+0000 7f56717fa640 1 -- 172.21.15.16:0/2074787531 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5668009c30 con 0x7f5674105320 2024-09-21T03:10:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.434+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56741a0760 con 0x7f5674105320 2024-09-21T03:10:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.434+0000 7f56717fa640 1 -- 172.21.15.16:0/2074787531 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680047a0 con 0x7f5674105320 2024-09-21T03:10:54.437 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.434+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56741a6ec0 con 0x7f5674105320 2024-09-21T03:10:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.436+0000 7f5680800740 1 Processor -- start 2024-09-21T03:10:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.436+0000 7f5680800740 1 -- start start 2024-09-21T03:10:54.438 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.436+0000 7f56717fa640 1 -- 172.21.15.16:0/2074787531 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f5668004020 con 0x7f5674105320 2024-09-21T03:10:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840c9b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d01390 0x5612840ca0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x56128403fe50 con 0x561283d22070 2024-09-21T03:10:54.439 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x56128402f280 con 0x561283cf91b0 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5680800740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x561284033090 con 0x561283d01390 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47076/0 (socket says 172.21.15.16:47076) 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 -- 172.21.15.16:0/1438987767 learned_addr learned my addr 172.21.15.16:0/1438987767 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5673fff640 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840c9b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f56737fe640 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d01390 0x5612840ca0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 -- 172.21.15.16:0/1438987767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d01390 msgr2=0x5612840ca0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d01390 0x5612840ca0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 -- 172.21.15.16:0/1438987767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 msgr2=0x5612840c9b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840c9b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 -- 172.21.15.16:0/1438987767 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561283d8d650 con 0x561283d22070 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.437+0000 7f5678829640 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f566c009f70 tx=0x7f566c010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=604543c945b71e18 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f56717fa640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 0x7f565c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f565a7fc640 1 -- 172.21.15.16:0/1438987767 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c016070 con 0x561283d22070 2024-09-21T03:10:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f565a7fc640 1 -- 172.21.15.16:0/1438987767 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c004d20 con 0x561283d22070 2024-09-21T03:10:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f5673fff640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 0x7f565c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f56717fa640 1 -- 172.21.15.16:0/2074787531 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f56680e7330 con 0x7f5674105320 2024-09-21T03:10:54.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.438+0000 7f565a7fc640 1 -- 172.21.15.16:0/1438987767 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566c0127d0 con 0x561283d22070 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- 172.21.15.16:0/1438987767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 msgr2=0x5612840c9650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5673fff640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 0x7f565c0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5664000c00 tx=0x7f5664031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f566c009f70 tx=0x7f566c010800 comp rx=0 tx=0).stop 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- 172.21.15.16:0/1438987767 shutdown_connections 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d01390 0x5612840ca0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840c9b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 --2- 172.21.15.16:0/1438987767 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d22070 0x5612840c9650 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- 172.21.15.16:0/1438987767 >> 172.21.15.16:0/1438987767 conn(0x561283d16460 msgr2=0x5612840c7540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- 172.21.15.16:0/1438987767 shutdown_connections 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- 172.21.15.16:0/1438987767 wait complete. 2024-09-21T03:10:54.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.439+0000 7f5680800740 1 -- start start 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 Processor -- start 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 0x5612840d9590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d9ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x561283e90700 con 0x561283cf91b0 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x561283fca110 con 0x561283d01390 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x5612840dc1e0 con 0x561283d22070 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5680800740 10 client.0 _ll_register_callbacks cb 0x561283fe3990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47092/0 (socket says 172.21.15.16:47092) 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5673fff640 1 -- 172.21.15.16:0/1637871135 learned_addr learned my addr 172.21.15.16:0/1637871135 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f56737fe640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 0x5612840d9590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.440+0000 7f5678829640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d9ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 -- 172.21.15.16:0/1637871135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 msgr2=0x5612840d9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d9ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 -- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 msgr2=0x5612840d9590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 0x5612840d9590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 -- 172.21.15.16:0/1637871135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561283ff7c40 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56737fe640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 0x5612840d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5673fff640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f567406c940 tx=0x7f5664009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc1fc6cb43ecb98c server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56537fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f566404d070 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f5664004450 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f566c004d20 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f5664059050 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5680800740 10 client.16605 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x5612840e7e60 con 0x561283cf91b0 2024-09-21T03:10:54.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.441+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f566c004d20 con 0x561283cf91b0 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5664004c80 con 0x561283cf91b0 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 10 client.16605 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 10 client.16605 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 20 client.16605 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561283ce09f0 0x5612840eb3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 4 client.16605 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561284041da0 con 0x561283ce09f0 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f56737fe640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561283ce09f0 0x5612840eb3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f56737fe640 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561283ce09f0 0x5612840eb3f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b76b2562d922e54 server_cookie=b22ea61be07f1c0e in_seq=0 out_seq=0 2024-09-21T03:10:54.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.442+0000 7f56537fe640 10 client.16605 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:54.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f56537fe640 1 -- 172.21.15.16:0/1637871135 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x561284041da0 con 0x561283ce09f0 2024-09-21T03:10:54.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f56537fe640 10 client.16605 handle_command_reply: tid=0 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 client.16605 shutdown 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 msgr2=0x5612840d9050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f567406c940 tx=0x7f5664009260 comp rx=0 tx=0).stop 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 shutdown_connections 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561283ce09f0 0x5612840eb3f0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d9ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d01390 0x5612840d9590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f56537fe640 0 client.16605 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 --2- 172.21.15.16:0/1637871135 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840d9050 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 >> 172.21.15.16:0/1637871135 conn(0x561283d16460 msgr2=0x5612840cb040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 shutdown_connections 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.443+0000 7f5680800740 1 -- 172.21.15.16:0/1637871135 wait complete. 2024-09-21T03:10:54.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.444+0000 7f5680800740 20 client.16605 trim_cache size 0 max 16384 2024-09-21T03:10:54.463 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 Processor -- start 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 -- start start 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840caf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d6780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ab500 con 0x561283d01390 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56741add60 con 0x561283cf91b0 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5680800740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56741abb50 con 0x561283d22070 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f56737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f56737fe640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47098/0 (socket says 172.21.15.16:47098) 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f56737fe640 1 -- 172.21.15.16:0/3025383117 learned_addr learned my addr 172.21.15.16:0/3025383117 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.461+0000 7f5673fff640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840caf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f5678829640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.464 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 -- 172.21.15.16:0/3025383117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 msgr2=0x5612840d6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 -- 172.21.15.16:0/3025383117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 msgr2=0x5612840caf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840caf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 -- 172.21.15.16:0/3025383117 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612840ca440 con 0x561283d01390 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f5673fff640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f56737fe640 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f5668094040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40a6d97a1185942e server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f5659ffb640 1 -- 172.21.15.16:0/3025383117 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680acac0 con 0x561283d01390 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f5659ffb640 1 -- 172.21.15.16:0/3025383117 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56680d0040 con 0x561283d01390 2024-09-21T03:10:54.465 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.462+0000 7f5659ffb640 1 -- 172.21.15.16:0/3025383117 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56680ac8e0 con 0x561283d01390 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- 172.21.15.16:0/3025383117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 msgr2=0x5612840d6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f5668005fd0 tx=0x7f5668094040 comp rx=0 tx=0).stop 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- 172.21.15.16:0/3025383117 shutdown_connections 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283d01390 0x5612840d6240 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283d22070 0x5612840d6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 --2- 172.21.15.16:0/3025383117 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283cf91b0 0x5612840caf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- 172.21.15.16:0/3025383117 >> 172.21.15.16:0/3025383117 conn(0x561283d16460 msgr2=0x5612840cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- 172.21.15.16:0/3025383117 shutdown_connections 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- 172.21.15.16:0/3025383117 wait complete. 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.463+0000 7f5680800740 1 -- start start 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 Processor -- start 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d22070 0x5612840dfca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.469 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283cb5da0 0x561284154dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ac220 con 0x561283cf91b0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ad690 con 0x561283d22070 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f56741ac8f0 con 0x561283cb5da0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5680800740 10 client.0 _ll_register_callbacks cb 0x7f5664032ef0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:47108/0 (socket says 172.21.15.16:47108) 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 -- 172.21.15.16:0/800986196 learned_addr learned my addr 172.21.15.16:0/800986196 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5678829640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283cb5da0 0x561284154dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f56737fe640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d22070 0x5612840dfca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 -- 172.21.15.16:0/800986196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283cb5da0 msgr2=0x561284154dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283cb5da0 0x561284154dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 -- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d22070 msgr2=0x5612840dfca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d22070 0x5612840dfca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.470 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.464+0000 7f5673fff640 1 -- 172.21.15.16:0/800986196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612840e04f0 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5673fff640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5664003800 tx=0x7f5664003ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9db2545b56b9a240 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5664061070 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(87..87 src has 1..87) ==== 8536+0+0 (secure 0 0 0) 0x7f5664004450 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f56680d0040 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7f566404d050 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 10 client.16611 fetch_fsmap learned FSMap version 20 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x561284156350 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56680d0040 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7f5664064020 con 0x561283cf91b0 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 10 client.16611 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 10 client.16611 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 20 client.16611 populate_metadata read hostname 'smithi016' 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561284156b50 0x561284158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 4 client.16611 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:10:54.471 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.465+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56128403d850 con 0x561284156b50 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f56737fe640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561284156b50 0x561284158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f56737fe640 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561284156b50 0x561284158f70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100b93118f9ac9ab server_cookie=28aafd0640e7e2bf in_seq=0 out_seq=0 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f56537fe640 10 client.16611 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f56537fe640 1 -- 172.21.15.16:0/800986196 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5682 (crc 0 0 0) 0x56128403d850 con 0x561284156b50 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f56537fe640 10 client.16611 handle_command_reply: tid=0 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.466+0000 7f5680800740 1 client.16611 shutdown 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 msgr2=0x5612840df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f5664003800 tx=0x7f5664003ab0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 shutdown_connections 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x561284156b50 0x561284158f70 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x561283cb5da0 0x561284154dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x561283d22070 0x5612840dfca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 --2- 172.21.15.16:0/800986196 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x561283cf91b0 0x5612840df760 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 >> 172.21.15.16:0/800986196 conn(0x561283d16460 msgr2=0x5612840cf140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f56537fe640 0 client.16611 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:10:54.472 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 shutdown_connections 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 1 -- 172.21.15.16:0/800986196 wait complete. 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.467+0000 7f5680800740 20 client.16611 trim_cache size 0 max 16384 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "IDLE (waiting for more scrubs)", 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": { 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "baf763ad-6228-4e74-a645-aeb506b4e304": { 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tag": "baf763ad-6228-4e74-a645-aeb506b4e304", 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: "options": "recursive,force" 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:54.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "baf763ad-6228-4e74-a645-aeb506b4e304": { 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:53.028066+0000", 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "749c373e-c09d-4335-b4aa-d8d285c516ab": { 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:13.847478+0000", 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.474 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.475 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:10:54.476 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.477 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.478 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.479 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.480 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.481 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:10:54.482 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:10:54.483 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:10:54.484 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 msgr2=0x7f565c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 0x7f565c0b8c10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5664000c00 tx=0x7f5664031400 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 msgr2=0x7f567419aaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f566800d950 tx=0x7f566800de10 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 shutdown_connections 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f565c0b6700 0x7f565c0b8c10 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5674105320 0x7f567419aaa0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f567410de30 0x7f567419fd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 --2- 172.21.15.16:0/2074787531 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f56741049d0 0x7f567419a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 >> 172.21.15.16:0/2074787531 conn(0x7f5674079740 msgr2=0x7f56740ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 shutdown_connections 2024-09-21T03:10:54.485 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:10:54.468+0000 7f567a28c640 1 -- 172.21.15.16:0/2074787531 wait complete. 2024-09-21T03:10:54.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:54 smithi123 ceph-mon[34645]: scrub queued for path: / 2024-09-21T03:10:54.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:54 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:54.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:54 smithi123 ceph-mon[34645]: scrub summary: active paths [/] 2024-09-21T03:10:54.486 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:54 smithi123 ceph-mon[34645]: scrub summary: idle+waiting paths [/] 2024-09-21T03:10:55.194 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:55 smithi016 ceph-mon[31253]: pgmap v645: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:55.224 INFO:tasks.cephfs.filesystem:scrub status for tag:baf763ad-6228-4e74-a645-aeb506b4e304 - {'path': '/', 'tag': 'baf763ad-6228-4e74-a645-aeb506b4e304', 'options': 'recursive,force'} 2024-09-21T03:10:55.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:55 smithi018 ceph-mon[34587]: pgmap v645: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:55.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:55 smithi123 ceph-mon[34645]: pgmap v645: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 853 B/s rd, 1 op/s 2024-09-21T03:10:57.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:57 smithi018 ceph-mon[34587]: pgmap v646: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 3 op/s 2024-09-21T03:10:57.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:57 smithi123 ceph-mon[34645]: pgmap v646: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 3 op/s 2024-09-21T03:10:57.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:57 smithi016 ceph-mon[31253]: pgmap v646: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 3 op/s 2024-09-21T03:10:58.407 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:58 smithi016 ceph-mon[31253]: scrub summary: idle 2024-09-21T03:10:58.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:58 smithi018 ceph-mon[34587]: scrub summary: idle 2024-09-21T03:10:58.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:58 smithi123 ceph-mon[34645]: scrub summary: idle 2024-09-21T03:10:59.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:10:59 smithi018 ceph-mon[34587]: pgmap v647: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s 2024-09-21T03:10:59.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:10:59 smithi123 ceph-mon[34645]: pgmap v647: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s 2024-09-21T03:10:59.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:10:59 smithi016 ceph-mon[31253]: pgmap v647: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 3.5 KiB/s rd, 4 op/s 2024-09-21T03:11:01.331 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:01 smithi016 ceph-mon[31253]: pgmap v648: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s 2024-09-21T03:11:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:01 smithi018 ceph-mon[34587]: pgmap v648: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s 2024-09-21T03:11:01.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:01 smithi123 ceph-mon[34645]: pgmap v648: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 4.1 KiB/s rd, 4 op/s 2024-09-21T03:11:03.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:03 smithi018 ceph-mon[34587]: pgmap v649: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:03.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:03 smithi123 ceph-mon[34645]: pgmap v649: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:03.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:03 smithi016 ceph-mon[31253]: pgmap v649: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:05.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:05 smithi018 ceph-mon[34587]: pgmap v650: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:05.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:05 smithi123 ceph-mon[34645]: pgmap v650: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:05.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:05 smithi016 ceph-mon[31253]: pgmap v650: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 22 KiB/s wr, 19 op/s 2024-09-21T03:11:07.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:07 smithi018 ceph-mon[34587]: pgmap v651: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-09-21T03:11:07.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:07 smithi123 ceph-mon[34645]: pgmap v651: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-09-21T03:11:07.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:07 smithi016 ceph-mon[31253]: pgmap v651: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 14 KiB/s rd, 45 KiB/s wr, 21 op/s 2024-09-21T03:11:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:09 smithi018 ceph-mon[34587]: pgmap v652: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-21T03:11:09.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:09 smithi018 ceph-mon[34587]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:11:09.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:09 smithi123 ceph-mon[34645]: pgmap v652: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-21T03:11:09.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:09 smithi123 ceph-mon[34645]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:11:09.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:09 smithi016 ceph-mon[31253]: pgmap v652: 97 pgs: 97 active+clean; 50 GiB data, 155 GiB used, 917 GiB / 1.0 TiB avail; 13 KiB/s rd, 45 KiB/s wr, 20 op/s 2024-09-21T03:11:09.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:09 smithi016 ceph-mon[31253]: osdmap e88: 12 total, 12 up, 12 in 2024-09-21T03:11:11.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:11 smithi018 ceph-mon[34587]: pgmap v654: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 49 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:11:11.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:11 smithi123 ceph-mon[34645]: pgmap v654: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 49 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:11:11.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:11 smithi016 ceph-mon[31253]: pgmap v654: 97 pgs: 7 active+clean+snaptrim, 16 active+clean+snaptrim_wait, 74 active+clean; 49 GiB data, 153 GiB used, 920 GiB / 1.0 TiB avail; 14 KiB/s rd, 54 KiB/s wr, 22 op/s 2024-09-21T03:11:13.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:13 smithi018 ceph-mon[34587]: pgmap v655: 97 pgs: 21 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 43 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:13.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:13 smithi123 ceph-mon[34645]: pgmap v655: 97 pgs: 21 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 43 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:13.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:13 smithi016 ceph-mon[31253]: pgmap v655: 97 pgs: 21 active+clean+snaptrim, 33 active+clean+snaptrim_wait, 43 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:15.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:15 smithi018 ceph-mon[34587]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:15.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:15 smithi123 ceph-mon[34645]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:15.518 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:15 smithi016 ceph-mon[31253]: pgmap v656: 97 pgs: 23 active+clean+snaptrim, 38 active+clean+snaptrim_wait, 36 active+clean; 48 GiB data, 149 GiB used, 924 GiB / 1.0 TiB avail; 3.2 KiB/s rd, 27 KiB/s wr, 5 op/s 2024-09-21T03:11:17.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:17 smithi018 ceph-mon[34587]: pgmap v657: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 46 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 2.0 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:17.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:17 smithi123 ceph-mon[34645]: pgmap v657: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 46 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 2.0 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:17.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:17 smithi016 ceph-mon[31253]: pgmap v657: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 46 GiB data, 141 GiB used, 931 GiB / 1.0 TiB avail; 2.0 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:19.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:19 smithi018 ceph-mon[34587]: pgmap v658: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 45 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:19.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:19 smithi123 ceph-mon[34645]: pgmap v658: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 45 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:19.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:19 smithi016 ceph-mon[31253]: pgmap v658: 97 pgs: 21 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 42 active+clean; 45 GiB data, 140 GiB used, 933 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:11:21.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:21 smithi018 ceph-mon[34587]: pgmap v659: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 44 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 177 B/s wr, 2 op/s 2024-09-21T03:11:21.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:21 smithi123 ceph-mon[34645]: pgmap v659: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 44 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 177 B/s wr, 2 op/s 2024-09-21T03:11:21.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:21 smithi016 ceph-mon[31253]: pgmap v659: 97 pgs: 20 active+clean+snaptrim, 34 active+clean+snaptrim_wait, 43 active+clean; 44 GiB data, 137 GiB used, 935 GiB / 1.0 TiB avail; 1.4 KiB/s rd, 177 B/s wr, 2 op/s 2024-09-21T03:11:22.931 INFO:tasks.workunit.client.0.smithi016.stdout:stopping iogen 2024-09-21T03:11:22.931 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo 'stopping iogen' 2024-09-21T03:11:22.931 INFO:tasks.workunit.client.0.smithi016.stderr:+ ./iogen -k 2024-09-21T03:11:22.932 INFO:tasks.workunit.client.0.smithi016.stdout:OK 2024-09-21T03:11:22.933 INFO:tasks.workunit.client.0.smithi016.stderr:+ echo OK 2024-09-21T03:11:23.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:23 smithi018 ceph-mon[34587]: pgmap v660: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:11:23.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:23 smithi123 ceph-mon[34645]: pgmap v660: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:11:23.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:23 smithi016 ceph-mon[31253]: pgmap v660: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 43 GiB data, 133 GiB used, 940 GiB / 1.0 TiB avail; 1023 B/s rd, 170 B/s wr, 2 op/s 2024-09-21T03:11:24.954 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-09-21T03:11:24.954 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-09-21T03:11:25.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 scrub status 2024-09-21T03:11:25.247 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:25 smithi016 ceph-mon[31253]: pgmap v661: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 42 GiB data, 132 GiB used, 941 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:11:25.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:25 smithi018 ceph-mon[34587]: pgmap v661: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 42 GiB data, 132 GiB used, 941 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:11:25.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:25 smithi123 ceph-mon[34645]: pgmap v661: 97 pgs: 14 active+clean+snaptrim, 26 active+clean+snaptrim_wait, 57 active+clean; 42 GiB data, 132 GiB used, 941 GiB / 1.0 TiB avail; 938 B/s rd, 85 B/s wr, 1 op/s 2024-09-21T03:11:25.499 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:25.640 INFO:tasks.workunit:Stopping ['suites/iogen.sh'] on client.0... 2024-09-21T03:11:25.641 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-09-21T03:11:25.993 DEBUG:teuthology.parallel:result is None 2024-09-21T03:11:25.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:11:26.031 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-09-21T03:11:26.031 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-09-21T03:11:26.042 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-09-21T03:11:26.042 INFO:teuthology.task.exec:Executing custom commands... 2024-09-21T03:11:26.042 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:11:26.042 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-09-21T03:11:26.101 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-09-21T03:11:26.156 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2339927565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654110120 msgr2=0x7fb654112540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2339927565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654110120 0x7fb654112540 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009ca0 tx=0x7fb64c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2339927565 shutdown_connections 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2339927565 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654110120 0x7fb654112540 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2339927565 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb654076100 0x7fb65410fbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2339927565 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb6540757b0 0x7fb654075bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2339927565 >> 172.21.15.16:0/2339927565 conn(0x7fb65407b910 msgr2=0x7fb65407bd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2339927565 shutdown_connections 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.120+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2339927565 wait complete. 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 Processor -- start 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 -- start start 2024-09-21T03:11:26.157 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 0x7fb65419c6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654110120 0x7fb6541a1f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb654115c90 con 0x7fb654076100 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb654115b10 con 0x7fb6540757b0 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65b2ff640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb654115e10 con 0x7fb654110120 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40694/0 (socket says 172.21.15.16:40694) 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.121+0000 7fb65909b640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 0x7fb65419c6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 -- 172.21.15.16:0/2381154206 learned_addr learned my addr 172.21.15.16:0/2381154206 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65989c640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654110120 0x7fb6541a1f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 -- 172.21.15.16:0/2381154206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654110120 msgr2=0x7fb6541a1f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654110120 0x7fb6541a1f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.158 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 -- 172.21.15.16:0/2381154206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 msgr2=0x7fb65419c6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 0x7fb65419c6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 -- 172.21.15.16:0/2381154206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6541a2590 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65909b640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 0x7fb65419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.122+0000 7fb65889a640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c970 tx=0x7fb64400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.123+0000 7fb64a7fc640 1 -- 172.21.15.16:0/2381154206 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644014070 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.123+0000 7fb64a7fc640 1 -- 172.21.15.16:0/2381154206 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb644004440 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.123+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6541a2880 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.123+0000 7fb64a7fc640 1 -- 172.21.15.16:0/2381154206 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb644003e50 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.123+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6541a90a0 con 0x7fb654076100 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.125+0000 7fb661873740 1 Processor -- start 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 -- start start 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.159 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60ea60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e60e1be50 con 0x560e60add390 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560e60e0b280 con 0x560e60afe070 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb661873740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e60e0f090 con 0x560e60ad51b0 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb64a7fc640 1 -- 172.21.15.16:0/2381154206 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb644017070 con 0x7fb654076100 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65909b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:43638/0 (socket says 172.21.15.16:43638) 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65909b640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45632/0 (socket says 172.21.15.16:45632) 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 -- 172.21.15.16:0/2843105348 learned_addr learned my addr 172.21.15.16:0/2843105348 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65889a640 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60ea60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 -- 172.21.15.16:0/2843105348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 msgr2=0x560e60ea5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 -- 172.21.15.16:0/2843105348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 msgr2=0x560e60ea60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.160 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60ea60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.126+0000 7fb65989c640 1 -- 172.21.15.16:0/2843105348 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e60b69650 con 0x560e60afe070 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb65909b640 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb65989c640 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009920 tx=0x7fb64c009950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20b8960b6a23a50a server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb63f7fe640 1 -- 172.21.15.16:0/2843105348 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c031e30 con 0x560e60afe070 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb63f7fe640 1 -- 172.21.15.16:0/2843105348 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c036030 con 0x560e60afe070 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb63f7fe640 1 -- 172.21.15.16:0/2843105348 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c004280 con 0x560e60afe070 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb64a7fc640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 0x7fb6400b8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb65909b640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 0x7fb6400b8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.127+0000 7fb64a7fc640 1 -- 172.21.15.16:0/2381154206 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb6440e7130 con 0x7fb654076100 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 -- 172.21.15.16:0/2843105348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 msgr2=0x560e60ea5650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb65909b640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 0x7fb6400b8d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x560e60ea42d0 tx=0x7fb65000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb64c009920 tx=0x7fb64c009950 comp rx=0 tx=0).stop 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 -- 172.21.15.16:0/2843105348 shutdown_connections 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60ea60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.161 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 --2- 172.21.15.16:0/2843105348 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea5650 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 -- 172.21.15.16:0/2843105348 >> 172.21.15.16:0/2843105348 conn(0x560e60af2460 msgr2=0x560e60ea3540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 -- 172.21.15.16:0/2843105348 shutdown_connections 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.128+0000 7fb661873740 1 -- 172.21.15.16:0/2843105348 wait complete. 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 -- start start 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 Processor -- start 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60add390 0x560e60eb1190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60afe070 0x560e60eb16d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x560e60c6c700 con 0x560e60ad51b0 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x560e60da6110 con 0x560e60add390 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x560e60eb3c50 con 0x560e60afe070 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb661873740 10 client.0 _ll_register_callbacks cb 0x560e60dbf990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:11:26.162 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb65909b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb65909b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40710/0 (socket says 172.21.15.16:40710) 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.129+0000 7fb65909b640 1 -- 172.21.15.16:0/2669635928 learned_addr learned my addr 172.21.15.16:0/2669635928 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65989c640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60afe070 0x560e60eb16d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65889a640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60add390 0x560e60eb1190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 -- 172.21.15.16:0/2669635928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60afe070 msgr2=0x560e60eb16d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60afe070 0x560e60eb16d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 -- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60add390 msgr2=0x560e60eb1190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60add390 0x560e60eb1190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 -- 172.21.15.16:0/2669635928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e60dd3c40 con 0x560e60ad51b0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb65909b640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb650010930 tx=0x7fb650004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=943fd85f862fcdb7 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb63cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb65002c060 con 0x560e60ad51b0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb650017030 con 0x560e60ad51b0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c036030 con 0x560e60ad51b0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb65001fac0 con 0x560e60ad51b0 2024-09-21T03:11:26.163 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb661873740 10 client.16623 fetch_fsmap learned FSMap version 20 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.130+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560e60ebf930 con 0x560e60ad51b0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c036030 con 0x560e60ad51b0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb65001b390 con 0x560e60ad51b0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 10 client.16623 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 10 client.16623 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 20 client.16623 populate_metadata read hostname 'smithi016' 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60abc9f0 0x560e60ec2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 4 client.16623 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e60e1dda0 con 0x560e60abc9f0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.131+0000 7fb65889a640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60abc9f0 0x560e60ec2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.132+0000 7fb65889a640 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60abc9f0 0x560e60ec2d90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=211696509f734a02 server_cookie=85ac27d03c2546c8 in_seq=0 out_seq=0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.132+0000 7fb63cff9640 10 client.16623 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.132+0000 7fb63cff9640 1 -- 172.21.15.16:0/2669635928 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x560e60e1dda0 con 0x560e60abc9f0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.132+0000 7fb63cff9640 10 client.16623 handle_command_reply: tid=0 2024-09-21T03:11:26.164 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.132+0000 7fb661873740 1 client.16623 shutdown 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 msgr2=0x560e60eb0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7fb650010930 tx=0x7fb650004730 comp rx=0 tx=0).stop 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 shutdown_connections 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60abc9f0 0x560e60ec2d90 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60afe070 0x560e60eb16d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60add390 0x560e60eb1190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb63cff9640 0 client.16623 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 --2- 172.21.15.16:0/2669635928 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60ad51b0 0x560e60eb0c50 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 >> 172.21.15.16:0/2669635928 conn(0x560e60af2460 msgr2=0x560e60ea87e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 shutdown_connections 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 1 -- 172.21.15.16:0/2669635928 wait complete. 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.133+0000 7fb661873740 20 client.16623 trim_cache size 0 max 16384 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 Processor -- start 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 -- start start 2024-09-21T03:11:26.165 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea7040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb2880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541afa30 con 0x560e60add390 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b2290 con 0x560e60afe070 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb661873740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b0080 con 0x560e60ad51b0 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65989c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65989c640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:43652/0 (socket says 172.21.15.16:43652) 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65989c640 1 -- 172.21.15.16:0/2117944151 learned_addr learned my addr 172.21.15.16:0/2117944151 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65909b640 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65989c640 1 -- 172.21.15.16:0/2117944151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 msgr2=0x560e60ea7040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.151+0000 7fb65989c640 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea7040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb65989c640 1 -- 172.21.15.16:0/2117944151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 msgr2=0x560e60eb2880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb65989c640 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.166 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb65989c640 1 -- 172.21.15.16:0/2117944151 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e60ebd5e0 con 0x560e60afe070 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb65989c640 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005cc0 tx=0x7fb64c003ea0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f79a8741a5f1396 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb63effd640 1 -- 172.21.15.16:0/2117944151 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c055020 con 0x560e60afe070 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb63effd640 1 -- 172.21.15.16:0/2117944151 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c036040 con 0x560e60afe070 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.152+0000 7fb63effd640 1 -- 172.21.15.16:0/2117944151 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb64c04a040 con 0x560e60afe070 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 -- 172.21.15.16:0/2117944151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 msgr2=0x560e60ea2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb64c005cc0 tx=0x7fb64c003ea0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 -- 172.21.15.16:0/2117944151 shutdown_connections 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb2880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60ea2b70 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 --2- 172.21.15.16:0/2117944151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60ea7040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 -- 172.21.15.16:0/2117944151 >> 172.21.15.16:0/2117944151 conn(0x560e60af2460 msgr2=0x560e60eb2170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.153+0000 7fb661873740 1 -- 172.21.15.16:0/2117944151 shutdown_connections 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.154+0000 7fb661873740 1 -- 172.21.15.16:0/2117944151 wait complete. 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.154+0000 7fb661873740 1 -- start start 2024-09-21T03:11:26.167 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.154+0000 7fb661873740 1 Processor -- start 2024-09-21T03:11:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60eb4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60eb5940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.171 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b0750 con 0x560e60add390 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b1bc0 con 0x560e60afe070 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb6541b0e20 con 0x560e60ad51b0 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb661873740 10 client.0 _ll_register_callbacks cb 0x7fb650005c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb65889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb65889a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40722/0 (socket says 172.21.15.16:40722) 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb65889a640 1 -- 172.21.15.16:0/2856203968 learned_addr learned my addr 172.21.15.16:0/2856203968 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.155+0000 7fb65909b640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60eb4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 -- 172.21.15.16:0/2856203968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 msgr2=0x560e60eb4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65989c640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60eb5940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60eb4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 -- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 msgr2=0x560e60eb5940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60eb5940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 -- 172.21.15.16:0/2856203968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e60f2c7d0 con 0x560e60add390 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65989c640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60eb5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:11:26.172 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65909b640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60eb4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb65889a640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb6440aced0 tx=0x7fb6440045f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccce8d2465f1dc9f server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb63cff9640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6440ac6b0 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb6440940d0 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.156+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb64c036040 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb6440aca50 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 10 client.16626 fetch_fsmap learned FSMap version 20 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x560e60f2daa0 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb64c036040 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb6440980d0 con 0x560e60add390 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 10 client.16626 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 10 client.16626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 20 client.16626 populate_metadata read hostname 'smithi016' 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60a91da0 0x560e60f301e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 4 client.16626 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-09-21T03:11:26.173 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.157+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560e60e19850 con 0x560e60a91da0 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb65909b640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60a91da0 0x560e60f301e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb65909b640 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60a91da0 0x560e60f301e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f6ed4308c9e6cb06 server_cookie=e6e14ac28664e2af in_seq=0 out_seq=0 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb63cff9640 10 client.16626 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb63cff9640 1 -- 172.21.15.16:0/2856203968 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+5486 (crc 0 0 0) 0x560e60e19850 con 0x560e60a91da0 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb63cff9640 10 client.16626 handle_command_reply: tid=0 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.158+0000 7fb661873740 1 client.16626 shutdown 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 msgr2=0x560e60eb5400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7fb6440aced0 tx=0x7fb6440045f0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 shutdown_connections 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x560e60a91da0 0x560e60f301e0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x560e60add390 0x560e60eb5400 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x560e60afe070 0x560e60eb5940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 --2- 172.21.15.16:0/2856203968 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560e60ad51b0 0x560e60eb4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.174 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 >> 172.21.15.16:0/2856203968 conn(0x560e60af2460 msgr2=0x560e60ea6e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb63cff9640 0 client.16626 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 shutdown_connections 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 1 -- 172.21.15.16:0/2856203968 wait complete. 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.159+0000 7fb661873740 20 client.16626 trim_cache size 0 max 16384 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "status": "no active scrubs running", 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "scrubs": {}, 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "scrub_stats": { 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "baf763ad-6228-4e74-a645-aeb506b4e304": { 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:53.028066+0000", 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.175 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "749c373e-c09d-4335-b4aa-d8d285c516ab": { 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:10:13.847478+0000", 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "6465f833-9841-4762-b989-30194f9d784e": { 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:09:34.076902+0000", 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.176 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "6acce003-5dce-479a-a188-c4c69ecf4f2f": { 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:53.882327+0000", 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "f18ec32e-7c06-4600-a70e-b062bbda174f": { 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:10.734784+0000", 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.177 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "2babef10-ec09-4cdf-987d-a41606a60c7f": { 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:08:00.240896+0000", 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "c2342f81-cbb6-415e-bcd5-bab15372fa32": { 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:19.884235+0000", 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.178 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "4f883922-f7c1-4b58-a99c-95533f769ca1": { 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:12.302325+0000", 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "bf89f1c3-0bf1-4a04-bd60-efd71bad1967": { 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:00:59.287769+0000", 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.179 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "fa8044a9-2474-4d92-84ff-5d4668e8b0cf": { 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:54.224701+0000", 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "15fd7ce4-15fb-4adf-a38a-09586ce5da82": { 2024-09-21T03:11:26.180 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:01:44.502236+0000", 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "d4145247-6bd4-4fe4-a89f-639a0a34ac56": { 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:05:37.162493+0000", 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "66226652-4934-4aed-bd6e-225c12fed8ba": { 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:07:17.884349+0000", 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "e2f8ab2e-228d-4686-8d9f-164a925af15d": { 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:02:27.393729+0000", 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.182 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "dd86dc76-816b-45f1-90ea-fc274a18fddc": { 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:03:10.784215+0000", 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "4f321092-7d1c-470d-ad98-0858877c6a2e": { 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:33.756838+0000", 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.183 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "3b241a9e-e2ba-4add-bf02-6d772b7c0821": { 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:39.397170+0000", 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "7357d59f-9636-4aa8-bd7f-5c495426dcb6": { 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:04:52.536015+0000", 2024-09-21T03:11:26.184 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "4c602931-953b-4cb1-8773-62796cf5e259": { 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "start_time": "2024-09-21T03:06:20.129991+0000", 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "path": "/", 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_started": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_passed": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_failed": 0, 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: "uninline_skipped": 0 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:26.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 msgr2=0x7fb6400b8d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 0x7fb6400b8d60 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x560e60ea42d0 tx=0x7fb65000b390 comp rx=0 tx=0).stop 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 msgr2=0x7fb65419cc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb64400c970 tx=0x7fb64400ce30 comp rx=0 tx=0).stop 2024-09-21T03:11:26.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 shutdown_connections 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb6400b6850 0x7fb6400b8d60 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb654076100 0x7fb65419cc80 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb654110120 0x7fb6541a1f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 --2- 172.21.15.16:0/2381154206 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb6540757b0 0x7fb65419c6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 >> 172.21.15.16:0/2381154206 conn(0x7fb65407b910 msgr2=0x7fb654109cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 shutdown_connections 2024-09-21T03:11:26.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.160+0000 7fb65b2ff640 1 -- 172.21.15.16:0/2381154206 wait complete. 2024-09-21T03:11:26.188 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 -- 172.21.15.16:0/23517389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f01029b0 msgr2=0x7ff0f0102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 --2- 172.21.15.16:0/23517389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f01029b0 0x7ff0f0102dc0 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0009900 tx=0x7ff0e002a410 comp rx=0 tx=0).stop 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 -- 172.21.15.16:0/23517389 shutdown_connections 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 --2- 172.21.15.16:0/23517389 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f0108c30 0x7ff0f010f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 --2- 172.21.15.16:0/23517389 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 0x7ff0f01086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 --2- 172.21.15.16:0/23517389 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f01029b0 0x7ff0f0102dc0 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 -- 172.21.15.16:0/23517389 >> 172.21.15.16:0/23517389 conn(0x7ff0f007a740 msgr2=0x7ff0f007ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 -- 172.21.15.16:0/23517389 shutdown_connections 2024-09-21T03:11:26.348 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.345+0000 7ff0f674a640 1 -- 172.21.15.16:0/23517389 wait complete. 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 Processor -- start 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 -- start start 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f01029b0 0x7ff0f019a410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 0x7ff0f019a9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0113a10 con 0x7ff0f0108c30 2024-09-21T03:11:26.350 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0113890 con 0x7ff0f01082e0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f674a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7ff0f0113b90 con 0x7ff0f01029b0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f4ce7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f4ce7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40752/0 (socket says 172.21.15.16:40752) 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.346+0000 7ff0f4ce7640 1 -- 172.21.15.16:0/2235541144 learned_addr learned my addr 172.21.15.16:0/2235541144 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0effff640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f01029b0 0x7ff0f019a410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0ef7fe640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 0x7ff0f019a9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 -- 172.21.15.16:0/2235541144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f01029b0 msgr2=0x7ff0f019a410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f01029b0 0x7ff0f019a410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 -- 172.21.15.16:0/2235541144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 msgr2=0x7ff0f019a9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 0x7ff0f019a9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 -- 172.21.15.16:0/2235541144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0f01a0340 con 0x7ff0f0108c30 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f4ce7640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800eaa0 tx=0x7ff0e800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e800cdb0 con 0x7ff0f0108c30 2024-09-21T03:11:26.351 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0f01a0630 con 0x7ff0f0108c30 2024-09-21T03:11:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e8005020 con 0x7ff0f0108c30 2024-09-21T03:11:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e80116d0 con 0x7ff0f0108c30 2024-09-21T03:11:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.347+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0f01a6dc0 con 0x7ff0f0108c30 2024-09-21T03:11:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.348+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0f0102dc0 con 0x7ff0f0108c30 2024-09-21T03:11:26.352 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.350+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7ff0e80051d0 con 0x7ff0f0108c30 2024-09-21T03:11:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.351+0000 7ff0ed7fa640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 0x7ff0d80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.351+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7ff0e80e7dd0 con 0x7ff0f0108c30 2024-09-21T03:11:26.353 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.351+0000 7ff0effff640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 0x7ff0d80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.354 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.352+0000 7ff0effff640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 0x7ff0d80b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0031440 tx=0x7ff0e0005d50 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.356 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.354+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7ff0e8016030 con 0x7ff0f0108c30 2024-09-21T03:11:26.574 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.572+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff0f0063800 con 0x7ff0d80b6680 2024-09-21T03:11:26.584 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.581+0000 7ff0ed7fa640 1 -- 172.21.15.16:0/2235541144 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7ff0f0063800 con 0x7ff0d80b6680 2024-09-21T03:11:26.584 INFO:teuthology.orchestra.run.smithi016.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-09-21T00:00:00", "created": "2024-09-21T03:00:11", "first": "2024-09-21T03:01:00", "last": "2024-09-21T03:11:00", "last_pruned": "2024-09-21T03:11:00", "created_count": 11, "pruned_count": 5, "active": true} 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 msgr2=0x7ff0d80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 0x7ff0d80b8b90 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7ff0e0031440 tx=0x7ff0e0005d50 comp rx=0 tx=0).stop 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 msgr2=0x7ff0f019fc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7ff0e800eaa0 tx=0x7ff0e800ef60 comp rx=0 tx=0).stop 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 shutdown_connections 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7ff0d80b6680 0x7ff0d80b8b90 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.586 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7ff0f0108c30 0x7ff0f019fc40 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7ff0f01082e0 0x7ff0f019a9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 --2- 172.21.15.16:0/2235541144 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0f01029b0 0x7ff0f019a410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 >> 172.21.15.16:0/2235541144 conn(0x7ff0f007a740 msgr2=0x7ff0f0106390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 shutdown_connections 2024-09-21T03:11:26.587 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.583+0000 7ff0f674a640 1 -- 172.21.15.16:0/2235541144 wait complete. 2024-09-21T03:11:26.598 DEBUG:teuthology.orchestra.run.smithi016:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-09-21T03:11:26.716 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 -- 172.21.15.16:0/4227826481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b8108c30 msgr2=0x7fe4b810f460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/4227826481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b8108c30 0x7fe4b810f460 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fe4b0009ca0 tx=0x7fe4b002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 -- 172.21.15.16:0/4227826481 shutdown_connections 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/4227826481 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b8108c30 0x7fe4b810f460 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/4227826481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81082e0 0x7fe4b81086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/4227826481 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b81029b0 0x7fe4b8102dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 -- 172.21.15.16:0/4227826481 >> 172.21.15.16:0/4227826481 conn(0x7fe4b807a740 msgr2=0x7fe4b807ab50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 -- 172.21.15.16:0/4227826481 shutdown_connections 2024-09-21T03:11:26.717 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 -- 172.21.15.16:0/4227826481 wait complete. 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.714+0000 7fe4bdd72640 1 Processor -- start 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 -- start start 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81029b0 0x7fe4b819b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b8108c30 0x7fe4b819bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b81128f0 con 0x7fe4b81082e0 2024-09-21T03:11:26.719 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8112770 con 0x7fe4b8108c30 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4bdd72640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fe4b8112a70 con 0x7fe4b81029b0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40778/0 (socket says 172.21.15.16:40778) 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 -- 172.21.15.16:0/1741635612 learned_addr learned my addr 172.21.15.16:0/1741635612 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4b7fff640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b8108c30 0x7fe4b819bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4b77fe640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81029b0 0x7fe4b819b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 -- 172.21.15.16:0/1741635612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81029b0 msgr2=0x7fe4b819b1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81029b0 0x7fe4b819b1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.715+0000 7fe4aeffd640 1 -- 172.21.15.16:0/1741635612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b8108c30 msgr2=0x7fe4b819bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4aeffd640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b8108c30 0x7fe4b819bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4aeffd640 1 -- 172.21.15.16:0/1741635612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe4b81a6430 con 0x7fe4b81082e0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4aeffd640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9c0 tx=0x7fe4a000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a000ccc0 con 0x7fe4b81082e0 2024-09-21T03:11:26.720 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe4a00054d0 con 0x7fe4b81082e0 2024-09-21T03:11:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4b81a6720 con 0x7fe4b81082e0 2024-09-21T03:11:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe4a0010430 con 0x7fe4b81082e0 2024-09-21T03:11:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.716+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4b81a6c20 con 0x7fe4b81082e0 2024-09-21T03:11:26.721 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.717+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4b8102dc0 con 0x7fe4b81082e0 2024-09-21T03:11:26.722 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.720+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fe4a0005000 con 0x7fe4b81082e0 2024-09-21T03:11:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.721+0000 7fe4b57fa640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 0x7fe4a40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.721+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fe4a00e7d10 con 0x7fe4b81082e0 2024-09-21T03:11:26.723 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.721+0000 7fe4b77fe640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 0x7fe4a40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:26.724 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.722+0000 7fe4b77fe640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 0x7fe4a40b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4b806b740 tx=0x7fe4a8005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:26.726 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.724+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fe4a0056710 con 0x7fe4b81082e0 2024-09-21T03:11:26.940 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.938+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 --> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7fe4b8063800 con 0x7fe4a40b6680 2024-09-21T03:11:26.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.947+0000 7fe4b57fa640 1 -- 172.21.15.16:0/1741635612 <== mgr.14226 v2:172.21.15.16:6800/3994309618 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7fe4b8063800 con 0x7fe4a40b6680 2024-09-21T03:11:26.949 INFO:teuthology.orchestra.run.smithi016.stdout:/ 1m 6m3h 2024-09-21T03:11:26.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 msgr2=0x7fe4a40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 0x7fe4a40b8b90 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe4b806b740 tx=0x7fe4a8005f90 comp rx=0 tx=0).stop 2024-09-21T03:11:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 msgr2=0x7fe4b819b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fe4a000e9c0 tx=0x7fe4a000ee80 comp rx=0 tx=0).stop 2024-09-21T03:11:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 shutdown_connections 2024-09-21T03:11:26.953 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fe4a40b6680 0x7fe4a40b8b90 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fe4b81082e0 0x7fe4b819b6e0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fe4b8108c30 0x7fe4b819bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 --2- 172.21.15.16:0/1741635612 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe4b81029b0 0x7fe4b819b1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 >> 172.21.15.16:0/1741635612 conn(0x7fe4b807a740 msgr2=0x7fe4b81062d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 shutdown_connections 2024-09-21T03:11:26.954 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:26.949+0000 7fe4bdd72640 1 -- 172.21.15.16:0/1741635612 wait complete. 2024-09-21T03:11:26.963 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-09-21T03:11:26.973 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-09-21T03:11:27.150 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-09-21T03:11:27.151 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:11:27.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:27 smithi016 ceph-mon[31253]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 41 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:11:27.409 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:27.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:27 smithi018 ceph-mon[34587]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 41 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:11:27.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:27 smithi123 ceph-mon[34645]: pgmap v662: 97 pgs: 4 active+clean+snaptrim, 10 active+clean+snaptrim_wait, 83 active+clean; 41 GiB data, 127 GiB used, 946 GiB / 1.0 TiB avail; 1.3 KiB/s rd, 85 B/s wr, 2 op/s 2024-09-21T03:11:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.940+0000 7f0117309640 1 -- 172.21.15.16:0/2147707286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 msgr2=0x7f0110107f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:27.943 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.940+0000 7f0117309640 1 --2- 172.21.15.16:0/2147707286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f0110107f80 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f010000aa10 tx=0x7f010002b720 comp rx=0 tx=0).stop 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 -- 172.21.15.16:0/2147707286 shutdown_connections 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 --2- 172.21.15.16:0/2147707286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f0110107f80 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 --2- 172.21.15.16:0/2147707286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f01101084c0 0x7f011010ecf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 --2- 172.21.15.16:0/2147707286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0110102230 0x7f0110102640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 -- 172.21.15.16:0/2147707286 >> 172.21.15.16:0/2147707286 conn(0x7f0110079e40 msgr2=0x7f011007a250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 -- 172.21.15.16:0/2147707286 shutdown_connections 2024-09-21T03:11:27.944 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.941+0000 7f0117309640 1 -- 172.21.15.16:0/2147707286 wait complete. 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 Processor -- start 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 -- start start 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 0x7f0110198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01101084c0 0x7f011019db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f0110111940 con 0x7f0110107b70 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f01101117c0 con 0x7f0110102230 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f0117309640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0110111ac0 con 0x7f01101084c0 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f01148a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f01148a4640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40800/0 (socket says 172.21.15.16:40800) 2024-09-21T03:11:27.946 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f01148a4640 1 -- 172.21.15.16:0/1002172962 learned_addr learned my addr 172.21.15.16:0/1002172962 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f01150a5640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 0x7f0110198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.942+0000 7f01158a6640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01101084c0 0x7f011019db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 -- 172.21.15.16:0/1002172962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01101084c0 msgr2=0x7f011019db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01101084c0 0x7f011019db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 -- 172.21.15.16:0/1002172962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 msgr2=0x7f0110198330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 0x7f0110198330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 -- 172.21.15.16:0/1002172962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f011019e2a0 con 0x7f0110107b70 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01150a5640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 0x7f0110198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01148a4640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0100006010 tx=0x7f01000042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100040070 con 0x7f0110107b70 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f011019e530 con 0x7f0110107b70 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f010003c460 con 0x7f0110107b70 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0100036030 con 0x7f0110107b70 2024-09-21T03:11:27.947 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.943+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01101a4cf0 con 0x7f0110107b70 2024-09-21T03:11:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.944+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0110102640 con 0x7f0110107b70 2024-09-21T03:11:27.948 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.946+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0100043070 con 0x7f0110107b70 2024-09-21T03:11:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.947+0000 7f01067fc640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 0x7f00fc0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.947+0000 7f01150a5640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 0x7f00fc0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:27.949 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.947+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7f01001097a0 con 0x7f0110107b70 2024-09-21T03:11:27.950 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.947+0000 7f01150a5640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 0x7f00fc0b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f010c005fd0 tx=0x7f010c005ea0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:27.952 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:27.950+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f01000cfcb0 con 0x7f0110107b70 2024-09-21T03:11:28.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.218+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0110199460 con 0x7f0110107b70 2024-09-21T03:11:28.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.220+0000 7f01067fc640 1 -- 172.21.15.16:0/1002172962 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f01000d4090 con 0x7f0110107b70 2024-09-21T03:11:28.223 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:28.223 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:11:28.223 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:11:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.225+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 msgr2=0x7f00fc0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.225+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 0x7f00fc0b8b90 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f010c005fd0 tx=0x7f010c005ea0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.225+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 msgr2=0x7f01101988d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:28.228 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.225+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0100006010 tx=0x7f01000042e0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 shutdown_connections 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f00fc0b6680 0x7f00fc0b8b90 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0110107b70 0x7f01101988d0 secure :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f0100006010 tx=0x7f01000042e0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01101084c0 0x7f011019db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 --2- 172.21.15.16:0/1002172962 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0110102230 0x7f0110198330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:28.229 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.226+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 >> 172.21.15.16:0/1002172962 conn(0x7f0110079e40 msgr2=0x7f0110105c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:28.230 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.227+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 shutdown_connections 2024-09-21T03:11:28.231 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:28.228+0000 7f0117309640 1 -- 172.21.15.16:0/1002172962 wait complete. 2024-09-21T03:11:28.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:28 smithi018 ceph-mon[34587]: from='client.16632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:28.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:28 smithi018 ceph-mon[34587]: from='client.16638 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:28.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:28 smithi123 ceph-mon[34645]: from='client.16632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:28.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:28 smithi123 ceph-mon[34645]: from='client.16638 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:28.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:28 smithi016 ceph-mon[31253]: from='client.16632 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:28.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:28 smithi016 ceph-mon[31253]: from='client.16638 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-09-21T03:11:29.080 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph tell mds.1:0 damage ls 2024-09-21T03:11:29.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:29 smithi018 ceph-mon[34587]: pgmap v663: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 41 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 2 op/s 2024-09-21T03:11:29.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:29 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1002172962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:29.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:29 smithi123 ceph-mon[34645]: pgmap v663: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 41 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 2 op/s 2024-09-21T03:11:29.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:29 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1002172962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:29.473 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:29.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:29 smithi016 ceph-mon[31253]: pgmap v663: 97 pgs: 2 active+clean+snaptrim, 5 active+clean+snaptrim_wait, 90 active+clean; 41 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 2 op/s 2024-09-21T03:11:29.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:29 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1002172962' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.178+0000 7fb097024640 1 -- 172.21.15.16:0/4102633071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb088095030 msgr2=0x7fb088095440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.180 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.178+0000 7fb097024640 1 --2- 172.21.15.16:0/4102633071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb088095030 0x7fb088095440 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fb080009900 tx=0x7fb08002a3f0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 -- 172.21.15.16:0/4102633071 shutdown_connections 2024-09-21T03:11:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 --2- 172.21.15.16:0/4102633071 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809b2c0 0x7fb0880a1af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 --2- 172.21.15.16:0/4102633071 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb08809a970 0x7fb08809ad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.181 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 --2- 172.21.15.16:0/4102633071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb088095030 0x7fb088095440 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 -- 172.21.15.16:0/4102633071 >> 172.21.15.16:0/4102633071 conn(0x7fb08800cd80 msgr2=0x7fb08800d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 -- 172.21.15.16:0/4102633071 shutdown_connections 2024-09-21T03:11:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.180+0000 7fb097024640 1 -- 172.21.15.16:0/4102633071 wait complete. 2024-09-21T03:11:30.182 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.181+0000 7fb097024640 1 Processor -- start 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.181+0000 7fb097024640 1 -- start start 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb088095030 0x7fb08809def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08809b2c0 0x7fb08809e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880a4fc0 con 0x7fb08809a970 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880a4e40 con 0x7fb088095030 2024-09-21T03:11:30.183 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb097024640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb0880a5140 con 0x7fb08809b2c0 2024-09-21T03:11:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb095821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb095821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40814/0 (socket says 172.21.15.16:40814) 2024-09-21T03:11:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.182+0000 7fb095821640 1 -- 172.21.15.16:0/2564391114 learned_addr learned my addr 172.21.15.16:0/2564391114 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.183+0000 7fb095821640 1 -- 172.21.15.16:0/2564391114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08809b2c0 msgr2=0x7fb08809e970 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-09-21T03:11:30.184 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.183+0000 7fb095821640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08809b2c0 0x7fb08809e970 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.183+0000 7fb095821640 1 -- 172.21.15.16:0/2564391114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb088095030 msgr2=0x7fb08809def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.183+0000 7fb095821640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb088095030 0x7fb08809def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.183+0000 7fb095821640 1 -- 172.21.15.16:0/2564391114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb08813f200 con 0x7fb08809a970 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.184+0000 7fb095821640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e970 tx=0x7fb08c00ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.184+0000 7fb0877fe640 1 -- 172.21.15.16:0/2564391114 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c00cd80 con 0x7fb08809a970 2024-09-21T03:11:30.185 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.184+0000 7fb0877fe640 1 -- 172.21.15.16:0/2564391114 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08c0035f0 con 0x7fb08809a970 2024-09-21T03:11:30.186 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.185+0000 7fb0877fe640 1 -- 172.21.15.16:0/2564391114 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c010690 con 0x7fb08809a970 2024-09-21T03:11:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.186+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb08813f4f0 con 0x7fb08809a970 2024-09-21T03:11:30.187 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.186+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb08813f9b0 con 0x7fb08809a970 2024-09-21T03:11:30.191 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.189+0000 7fb09e7fa740 1 Processor -- start 2024-09-21T03:11:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.190+0000 7fb09e7fa740 1 -- start start 2024-09-21T03:11:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c40460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c40ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.193 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6bb6ea0 con 0x55a9b68701b0 2024-09-21T03:11:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6ba62d0 con 0x55a9b6878390 2024-09-21T03:11:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.191+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6baa0e0 con 0x55a9b6899070 2024-09-21T03:11:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.192+0000 7fb0877fe640 1 -- 172.21.15.16:0/2564391114 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb08c003aa0 con 0x7fb08809a970 2024-09-21T03:11:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.193+0000 7fb096022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.194 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.193+0000 7fb096022640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40822/0 (socket says 172.21.15.16:40822) 2024-09-21T03:11:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.193+0000 7fb096022640 1 -- 172.21.15.16:0/2645914726 learned_addr learned my addr 172.21.15.16:0/2645914726 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:30.195 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.193+0000 7fb096823640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c40460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.194+0000 7fb095821640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c40ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.194+0000 7fb096022640 1 -- 172.21.15.16:0/2645914726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 msgr2=0x55a9b6c40460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.194+0000 7fb096022640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c40460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.194+0000 7fb096022640 1 -- 172.21.15.16:0/2645914726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 msgr2=0x55a9b6c40ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.196 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.194+0000 7fb0877fe640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 0x7fb07c0b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb096022640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c40ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb096022640 1 -- 172.21.15.16:0/2645914726 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b6904650 con 0x55a9b68701b0 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb095821640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb096823640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 0x7fb07c0b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb0877fe640 1 -- 172.21.15.16:0/2564391114 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb08c0e7c70 con 0x7fb08809a970 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.195+0000 7fb096022640 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb08002a910 tx=0x7fb080000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e3111f5f660c5ca server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.196+0000 7fb084d9a640 1 -- 172.21.15.16:0/2645914726 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08003e030 con 0x55a9b68701b0 2024-09-21T03:11:30.197 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.196+0000 7fb084d9a640 1 -- 172.21.15.16:0/2645914726 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb080005020 con 0x55a9b68701b0 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.196+0000 7fb084d9a640 1 -- 172.21.15.16:0/2645914726 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb080035040 con 0x55a9b68701b0 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 -- 172.21.15.16:0/2645914726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 msgr2=0x55a9b6c409a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7fb08002a910 tx=0x7fb080000bc0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 -- 172.21.15.16:0/2645914726 shutdown_connections 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b68701b0 0x55a9b6c409a0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c40ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/2645914726 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c40460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.199 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.197+0000 7fb09e7fa740 1 -- 172.21.15.16:0/2645914726 >> 172.21.15.16:0/2645914726 conn(0x55a9b688d460 msgr2=0x55a9b6c3e350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.200 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.198+0000 7fb096823640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 0x7fb07c0b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb090051230 tx=0x7fb090068db0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.203+0000 7fb09e7fa740 1 -- 172.21.15.16:0/2645914726 shutdown_connections 2024-09-21T03:11:30.204 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.203+0000 7fb09e7fa740 1 -- 172.21.15.16:0/2645914726 wait complete. 2024-09-21T03:11:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.203+0000 7fb09e7fa740 1 -- start start 2024-09-21T03:11:30.205 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.203+0000 7fb09e7fa740 1 Processor -- start 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4ba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c4bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6a07700 con 0x55a9b6878390 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6b41160 con 0x55a9b68701b0 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x55a9b6c4ec30 con 0x55a9b6899070 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb09e7fa740 10 client.0 _ll_register_callbacks cb 0x55a9b6b5a9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb096823640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb095821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c4bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.207 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb096823640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:45748/0 (socket says 172.21.15.16:45748) 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.204+0000 7fb096823640 1 -- 172.21.15.16:0/1659998627 learned_addr learned my addr 172.21.15.16:0/1659998627 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096022640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4ba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 -- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 msgr2=0x55a9b6c4ba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4ba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 -- 172.21.15.16:0/1659998627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 msgr2=0x55a9b6c4bf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c4bf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 -- 172.21.15.16:0/1659998627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b6b6ec90 con 0x55a9b6899070 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb095821640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c4bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.205+0000 7fb096823640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb090000b60 tx=0x7fb090069570 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f0fbb137747469de server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.208 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.206+0000 7fb079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.206+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090081710 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.206+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb080005020 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.206+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mon.1 v2:172.21.15.123:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb0900818b0 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.207+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb09009c050 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.207+0000 7fb09e7fa740 10 client.26266 fetch_fsmap learned FSMap version 20 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.207+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a9b6c5a900 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.207+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mon.1 v2:172.21.15.123:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb080005020 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.207+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mon.1 v2:172.21.15.123:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb090097bf0 con 0x55a9b6899070 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 10 client.26266 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 10 client.26266 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 20 client.26266 populate_metadata read hostname 'smithi016' 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b68579f0 0x55a9b6c5e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.210 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 4 client.26266 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-09-21T03:11:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.208+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a9b6bb8df0 con 0x55a9b68579f0 2024-09-21T03:11:30.211 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.209+0000 7fb096022640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b68579f0 0x55a9b6c5e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.210+0000 7fb079ffb640 10 client.26266 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:30.212 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.210+0000 7fb096022640 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b68579f0 0x55a9b6c5e2e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d989941fc894a0e1 server_cookie=36a3097cb8fc707a in_seq=0 out_seq=0 2024-09-21T03:11:30.213 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.212+0000 7fb079ffb640 1 -- 172.21.15.16:0/1659998627 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+24444 (crc 0 0 0) 0x55a9b6bb8df0 con 0x55a9b68579f0 2024-09-21T03:11:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.212+0000 7fb079ffb640 10 client.26266 handle_command_reply: tid=0 2024-09-21T03:11:30.214 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.212+0000 7fb09e7fa740 1 client.26266 shutdown 2024-09-21T03:11:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.212+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 msgr2=0x55a9b6c4c4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.216 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.212+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fb090000b60 tx=0x7fb090069570 comp rx=0 tx=0).stop 2024-09-21T03:11:30.219 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 shutdown_connections 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b68579f0 0x55a9b6c5e2e0 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c4bf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c4c4b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1659998627 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4ba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb079ffb640 0 client.26266 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:30.221 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.217+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 >> 172.21.15.16:0/1659998627 conn(0x55a9b688d460 msgr2=0x55a9b6c435f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.218+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 shutdown_connections 2024-09-21T03:11:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.218+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1659998627 wait complete. 2024-09-21T03:11:30.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.218+0000 7fb09e7fa740 20 client.26266 trim_cache size 0 max 16384 2024-09-21T03:11:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.241+0000 7fb09e7fa740 1 Processor -- start 2024-09-21T03:11:30.243 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 -- start start 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4d910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c3d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb088146440 con 0x55a9b6878390 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb088148ca0 con 0x55a9b68701b0 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb088146a90 con 0x55a9b6899070 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb095821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb095821640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40832/0 (socket says 172.21.15.16:40832) 2024-09-21T03:11:30.244 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb095821640 1 -- 172.21.15.16:0/1807310561 learned_addr learned my addr 172.21.15.16:0/1807310561 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb096022640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.245 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.242+0000 7fb096823640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c3d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 -- 172.21.15.16:0/1807310561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 msgr2=0x55a9b6c3d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c3d980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 -- 172.21.15.16:0/1807310561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 msgr2=0x55a9b6c4d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 -- 172.21.15.16:0/1807310561 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a9b6c4ccf0 con 0x55a9b6878390 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb096823640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.243+0000 7fb095821640 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb08c003e20 tx=0x7fb08c0ad7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ccf157eabdd4773 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.244+0000 7fb07bfff640 1 -- 172.21.15.16:0/1807310561 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0ad190 con 0x55a9b6878390 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.244+0000 7fb07bfff640 1 -- 172.21.15.16:0/1807310561 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08c0ad530 con 0x55a9b6878390 2024-09-21T03:11:30.246 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.244+0000 7fb07bfff640 1 -- 172.21.15.16:0/1807310561 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb08c0037a0 con 0x55a9b6878390 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1807310561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 msgr2=0x55a9b6c41ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7fb08c003e20 tx=0x7fb08c0ad7f0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1807310561 shutdown_connections 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6878390 0x55a9b6c41ba0 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b6899070 0x55a9b6c3d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/1807310561 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b68701b0 0x55a9b6c4d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.245+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1807310561 >> 172.21.15.16:0/1807310561 conn(0x55a9b688d460 msgr2=0x55a9b6c41490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.246+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1807310561 shutdown_connections 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.246+0000 7fb09e7fa740 1 -- 172.21.15.16:0/1807310561 wait complete. 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.246+0000 7fb09e7fa740 1 -- start start 2024-09-21T03:11:30.248 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.246+0000 7fb09e7fa740 1 Processor -- start 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b68701b0 0x55a9b6c54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c4fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb088147160 con 0x55a9b6899070 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb0881485d0 con 0x55a9b6878390 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb088147830 con 0x55a9b68701b0 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb09e7fa740 10 client.0 _ll_register_callbacks cb 0x7fb0900514d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb096823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb096823640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40848/0 (socket says 172.21.15.16:40848) 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.247+0000 7fb096823640 1 -- 172.21.15.16:0/3187027412 learned_addr learned my addr 172.21.15.16:0/3187027412 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:30.251 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 -- 172.21.15.16:0/3187027412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b68701b0 msgr2=0x55a9b6c54960 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b68701b0 0x55a9b6c54960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 -- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 msgr2=0x55a9b6c4fe50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c4fe50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 -- 172.21.15.16:0/3187027412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a9b6cc7ce0 con 0x55a9b6899070 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb096823640 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb09006a550 tx=0x7fb090080840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb565896f64e41b6 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.248+0000 7fb079ffb640 10 client.0 ms_handle_connect on v2:172.21.15.16:3300/0 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb090081710 con 0x55a9b6899070 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb08c0ad530 con 0x55a9b6899070 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mon.0 v2:172.21.15.16:3300/0 2 ==== osd_map(88..88 src has 1..88) ==== 8580+0+0 (secure 0 0 0) 0x7fb0900818b0 con 0x55a9b6899070 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_get_version_reply(handle=1 version=20) ==== 24+0+0 (secure 0 0 0) 0x7fb0900b0050 con 0x55a9b6899070 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb09e7fa740 10 client.16662 fetch_fsmap learned FSMap version 20 2024-09-21T03:11:30.252 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.249+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,fsmap=20,monmap=4+}) -- 0x55a9b6cc8e30 con 0x55a9b6899070 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mon.0 v2:172.21.15.16:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb08c0ad530 con 0x55a9b6899070 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mon.0 v2:172.21.15.16:3300/0 5 ==== fsmap(e 20) ==== 8330+0+0 (secure 0 0 0) 0x7fb09009f040 con 0x55a9b6899070 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 10 client.16662 fetch_fsmap finished waiting for FSMap version 20 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 20 client.16662 populate_metadata read hostname 'smithi016' 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b682cda0 0x55a9b6ccb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:30.255 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 4 client.16662 mds_command: new command op to 24701 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.250+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 --> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a9b6bb48a0 con 0x55a9b682cda0 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.251+0000 7fb096022640 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b682cda0 0x55a9b6ccb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb096022640 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b682cda0 0x55a9b6ccb610 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f37cfa56d63eae0b server_cookie=6edbea16145ca651 in_seq=0 out_seq=0 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb079ffb640 10 client.16662 ms_handle_connect on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb079ffb640 1 -- 172.21.15.16:0/3187027412 <== mds.0 v2:172.21.15.18:6838/253530783 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a9b6bb48a0 con 0x55a9b682cda0 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb079ffb640 10 client.16662 handle_command_reply: tid=0 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb09e7fa740 1 client.16662 shutdown 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 msgr2=0x55a9b6c50560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.253+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7fb09006a550 tx=0x7fb090080840 comp rx=0 tx=0).stop 2024-09-21T03:11:30.256 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 shutdown_connections 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:6838/253530783,v1:172.21.15.18:6839/253530783] conn(0x55a9b682cda0 0x55a9b6ccb610 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x55a9b6899070 0x55a9b6c50560 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x55a9b6878390 0x55a9b6c4fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 --2- 172.21.15.16:0/3187027412 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a9b68701b0 0x55a9b6c54960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 >> 172.21.15.16:0/3187027412 conn(0x55a9b688d460 msgr2=0x55a9b6c4d650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.254+0000 7fb079ffb640 0 client.16662 ms_handle_reset on v2:172.21.15.18:6838/253530783 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.255+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 shutdown_connections 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.255+0000 7fb09e7fa740 1 -- 172.21.15.16:0/3187027412 wait complete. 2024-09-21T03:11:30.258 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.255+0000 7fb09e7fa740 20 client.16662 trim_cache size 0 max 16384 2024-09-21T03:11:30.259 INFO:teuthology.orchestra.run.smithi016.stdout:[] 2024-09-21T03:11:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.256+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 msgr2=0x7fb07c0b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.259 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.256+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 0x7fb07c0b8c10 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fb090051230 tx=0x7fb090068db0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.256+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 msgr2=0x7fb08809e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.256+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7fb08c00e970 tx=0x7fb08c00ee30 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 shutdown_connections 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb07c0b6700 0x7fb07c0b8c10 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb08809a970 0x7fb08809e430 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08809b2c0 0x7fb08809e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 --2- 172.21.15.16:0/2564391114 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb088095030 0x7fb08809def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 >> 172.21.15.16:0/2564391114 conn(0x7fb08800cd80 msgr2=0x7fb088093ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 shutdown_connections 2024-09-21T03:11:30.260 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:30.257+0000 7fb097024640 1 -- 172.21.15.16:0/2564391114 wait complete. 2024-09-21T03:11:31.307 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/git.ceph.com_ceph-c_1843221895adccfb3b692060251dd6a0183bc410/qa/tasks/thrasher.py", line 50, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-09-21T03:11:31.308 INFO:tasks.fwd_scrub:done joining 2024-09-21T03:11:31.308 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-09-21T03:11:31.319 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-09-21T03:11:31.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:31 smithi018 ceph-mon[34587]: pgmap v664: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-21T03:11:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 -- 172.21.15.16:0/2128825683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03281029b0 msgr2=0x7f0328102dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 --2- 172.21.15.16:0/2128825683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03281029b0 0x7f0328102dc0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f0324009900 tx=0x7f032402a410 comp rx=0 tx=0).stop 2024-09-21T03:11:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 -- 172.21.15.16:0/2128825683 shutdown_connections 2024-09-21T03:11:31.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 --2- 172.21.15.16:0/2128825683 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f0328108c30 0x7f032810f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 --2- 172.21.15.16:0/2128825683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 0x7f03281086f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 --2- 172.21.15.16:0/2128825683 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f03281029b0 0x7f0328102dc0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 -- 172.21.15.16:0/2128825683 >> 172.21.15.16:0/2128825683 conn(0x7f03280fe3b0 msgr2=0x7f0328100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 -- 172.21.15.16:0/2128825683 shutdown_connections 2024-09-21T03:11:31.447 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.444+0000 7f032f569640 1 -- 172.21.15.16:0/2128825683 wait complete. 2024-09-21T03:11:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 Processor -- start 2024-09-21T03:11:31.449 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 -- start start 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 0x7f03281998d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0328108c30 0x7f032819eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f03281128f0 con 0x7f0328108c30 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f0328112770 con 0x7f03281029b0 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.445+0000 7f032f569640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f0328112a70 con 0x7f03281082e0 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032db06640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0328108c30 0x7f032819eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.18:3300/0 says I am v2:172.21.15.16:43762/0 (socket says 172.21.15.16:43762) 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 -- 172.21.15.16:0/1120561603 learned_addr learned my addr 172.21.15.16:0/1120561603 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032cb04640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 0x7f03281998d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.450 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 -- 172.21.15.16:0/1120561603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 msgr2=0x7f03281998d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 0x7f03281998d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 -- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0328108c30 msgr2=0x7f032819eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0328108c30 0x7f032819eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 -- 172.21.15.16:0/1120561603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f032819f260 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032d305640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0324005940 tx=0x7f03240040c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f032403f030 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f03240044c0 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f032819f4f0 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0324035070 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.446+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03281a5ce0 con 0x7f03281029b0 2024-09-21T03:11:31.451 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.447+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0328102dc0 con 0x7f03281029b0 2024-09-21T03:11:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.449+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f0324042030 con 0x7f03281029b0 2024-09-21T03:11:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.450+0000 7f031e7fc640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 0x7f03140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.450+0000 7f032cb04640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 0x7f03140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.450+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f0324107ce0 con 0x7f03281029b0 2024-09-21T03:11:31.452 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.450+0000 7f032cb04640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 0x7f03140b8b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0328101c00 tx=0x7f0318005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:31.454 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.453+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f03240ce220 con 0x7f03281029b0 2024-09-21T03:11:31.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:31 smithi123 ceph-mon[34645]: pgmap v664: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-21T03:11:31.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:31 smithi016 ceph-mon[31253]: pgmap v664: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 23 KiB/s rd, 25 KiB/s wr, 30 op/s 2024-09-21T03:11:31.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.642+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f032810aa60 con 0x7f03281029b0 2024-09-21T03:11:31.645 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.644+0000 7f031e7fc640 1 -- 172.21.15.16:0/1120561603 <== mon.2 v2:172.21.15.18:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23217 (secure 0 0 0) 0x7f03240d2600 con 0x7f03281029b0 2024-09-21T03:11:31.646 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:31.646 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T03:11:31.297206+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:11:31.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 msgr2=0x7f03140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 0x7f03140b8b90 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f0328101c00 tx=0x7f0318005f90 comp rx=0 tx=0).stop 2024-09-21T03:11:31.648 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 msgr2=0x7f0328199330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f0324005940 tx=0x7f03240040c0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 shutdown_connections 2024-09-21T03:11:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f03140b6680 0x7f03140b8b90 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f0328108c30 0x7f032819eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.649 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03281082e0 0x7f03281998d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 --2- 172.21.15.16:0/1120561603 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f03281029b0 0x7f0328199330 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 >> 172.21.15.16:0/1120561603 conn(0x7f03280fe3b0 msgr2=0x7f0328107980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 shutdown_connections 2024-09-21T03:11:31.650 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.646+0000 7f032f569640 1 -- 172.21.15.16:0/1120561603 wait complete. 2024-09-21T03:11:31.659 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-09-21T02:53:55.521347+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-09-21T02:58:24.062466+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '75', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-09-21T02:58:26.393380+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'is_stretch_pool': False, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '88', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 17, 'snap_epoch': 88, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-09-21T03:11:31.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-09-21T03:11:31.777 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 -- 172.21.15.16:0/2950514787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109d60 msgr2=0x7f2700110590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 --2- 172.21.15.16:0/2950514787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109d60 0x7f2700110590 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f26f8009ca0 tx=0x7f26f802a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 -- 172.21.15.16:0/2950514787 shutdown_connections 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 --2- 172.21.15.16:0/2950514787 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109d60 0x7f2700110590 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 --2- 172.21.15.16:0/2950514787 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700109410 0x7f2700109820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 --2- 172.21.15.16:0/2950514787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700103ae0 0x7f2700103ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 -- 172.21.15.16:0/2950514787 >> 172.21.15.16:0/2950514787 conn(0x7f270007a8c0 msgr2=0x7f270007acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 -- 172.21.15.16:0/2950514787 shutdown_connections 2024-09-21T03:11:31.778 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.776+0000 7f27048cc640 1 -- 172.21.15.16:0/2950514787 wait complete. 2024-09-21T03:11:31.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 Processor -- start 2024-09-21T03:11:31.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 -- start start 2024-09-21T03:11:31.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700103ae0 0x7f270019a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.780 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700109d60 0x7f270019fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f2700113a20 con 0x7f2700109410 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f27001138a0 con 0x7f2700103ae0 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f27048cc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f2700113ba0 con 0x7f2700109d60 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fdd9b640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40894/0 (socket says 172.21.15.16:40894) 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fdd9b640 1 -- 172.21.15.16:0/2159356816 learned_addr learned my addr 172.21.15.16:0/2159356816 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fe59c640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700103ae0 0x7f270019a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fed9d640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700109d60 0x7f270019fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fdd9b640 1 -- 172.21.15.16:0/2159356816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700109d60 msgr2=0x7f270019fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.777+0000 7f26fdd9b640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700109d60 0x7f270019fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26fdd9b640 1 -- 172.21.15.16:0/2159356816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700103ae0 msgr2=0x7f270019a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.781 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26fdd9b640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700103ae0 0x7f270019a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26fdd9b640 1 -- 172.21.15.16:0/2159356816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f27001a0310 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26fdd9b640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d740 tx=0x7f26f400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f4017070 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f27001a0600 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26f400de50 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26f400f620 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.778+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f27001a6d90 con 0x7f2700109410 2024-09-21T03:11:31.782 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.779+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2700103ef0 con 0x7f2700109410 2024-09-21T03:11:31.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.781+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f26f401a070 con 0x7f2700109410 2024-09-21T03:11:31.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.781+0000 7f26ef7fe640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 0x7f26e80b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:31.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.781+0000 7f26fe59c640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 0x7f26e80b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:31.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.782+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f26f40e71d0 con 0x7f2700109410 2024-09-21T03:11:31.783 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.782+0000 7f26fe59c640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 0x7f26e80b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009b60 tx=0x7f26f0005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:31.786 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.784+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f26f40ad710 con 0x7f2700109410 2024-09-21T03:11:31.979 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.978+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f270010bbb0 con 0x7f2700109410 2024-09-21T03:11:31.980 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.978+0000 7f26ef7fe640 1 -- 172.21.15.16:0/2159356816 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f26f40b1af0 con 0x7f2700109410 2024-09-21T03:11:31.980 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 1 2024-09-21T03:11:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 msgr2=0x7f26e80b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.982 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 0x7f26e80b8b90 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f26f0009b60 tx=0x7f26f0005f90 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 msgr2=0x7f270019a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7f26f400d740 tx=0x7f26f400dc00 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 shutdown_connections 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f26e80b6680 0x7f26e80b8b90 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f2700109410 0x7f270019a980 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2700109d60 0x7f270019fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 --2- 172.21.15.16:0/2159356816 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f2700103ae0 0x7f270019a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 >> 172.21.15.16:0/2159356816 conn(0x7f270007a8c0 msgr2=0x7f2700105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 shutdown_connections 2024-09-21T03:11:31.983 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:31.981+0000 7f27048cc640 1 -- 172.21.15.16:0/2159356816 wait complete. 2024-09-21T03:11:31.994 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-09-21T03:11:32.111 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 -- 172.21.15.16:0/1974464737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c106b10 msgr2=0x7f6b7c1105a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1974464737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1105a0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f6b74009ca0 tx=0x7f6b7402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 -- 172.21.15.16:0/1974464737 shutdown_connections 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1974464737 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1105a0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1974464737 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c1061c0 0x7f6b7c1065d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1974464737 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c1023f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 -- 172.21.15.16:0/1974464737 >> 172.21.15.16:0/1974464737 conn(0x7f6b7c079e10 msgr2=0x7f6b7c07a220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 -- 172.21.15.16:0/1974464737 shutdown_connections 2024-09-21T03:11:32.112 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 -- 172.21.15.16:0/1974464737 wait complete. 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.110+0000 7f6b822f9640 1 Processor -- start 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 -- start start 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c1061c0 0x7f6b7c1034b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1039f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c113a50 con 0x7f6b7c101fe0 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c1138d0 con 0x7f6b7c106b10 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b822f9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6b7c113bd0 con 0x7f6b7c1061c0 2024-09-21T03:11:32.114 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40902/0 (socket says 172.21.15.16:40902) 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 -- 172.21.15.16:0/1804946991 learned_addr learned my addr 172.21.15.16:0/1804946991 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7b7fe640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c1061c0 0x7f6b7c1034b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b80896640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1039f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 -- 172.21.15.16:0/1804946991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c1061c0 msgr2=0x7f6b7c1034b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c1061c0 0x7f6b7c1034b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 -- 172.21.15.16:0/1804946991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c106b10 msgr2=0x7f6b7c1039f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1039f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.111+0000 7f6b7bfff640 1 -- 172.21.15.16:0/1804946991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b7c1a86c0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b7bfff640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9c0 tx=0x7f6b6c00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c00cd80 con 0x7f6b7c101fe0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b6c0035f0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b7c1a89b0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.115 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c010690 con 0x7f6b7c101fe0 2024-09-21T03:11:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.112+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b7c1a8eb0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.113+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6b7c1023f0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.116 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.114+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f6b6c004000 con 0x7f6b7c101fe0 2024-09-21T03:11:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.115+0000 7f6b797fa640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 0x7f6b640b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.115+0000 7f6b7b7fe640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 0x7f6b640b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.115+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f6b6c0e7ca0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.117 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.116+0000 7f6b7b7fe640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 0x7f6b640b8bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c101a50 tx=0x7f6b7000b410 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.121 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.119+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6b6c0ae1e0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.307+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6b7c02e640 con 0x7f6b7c101fe0 2024-09-21T03:11:32.309 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.308+0000 7f6b797fa640 1 -- 172.21.15.16:0/1804946991 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f6b6c0b25c0 con 0x7f6b7c101fe0 2024-09-21T03:11:32.309 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 32 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 msgr2=0x7f6b640b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 0x7f6b640b8bc0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f6b7c101a50 tx=0x7f6b7000b410 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 msgr2=0x7f6b7c102f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c00e9c0 tx=0x7f6b6c00ee80 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 shutdown_connections 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f6b640b66b0 0x7f6b640b8bc0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6b7c106b10 0x7f6b7c1039f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b7c1061c0 0x7f6b7c1034b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 --2- 172.21.15.16:0/1804946991 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6b7c101fe0 0x7f6b7c102f70 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 >> 172.21.15.16:0/1804946991 conn(0x7f6b7c079e10 msgr2=0x7f6b7c06a260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 shutdown_connections 2024-09-21T03:11:32.312 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.310+0000 7f6b822f9640 1 -- 172.21.15.16:0/1804946991 wait complete. 2024-09-21T03:11:32.323 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-09-21T03:11:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 -- 172.21.15.16:0/2350979654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 msgr2=0x7f17281086d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 --2- 172.21.15.16:0/2350979654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f17281086d0 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f171c009900 tx=0x7f171c02a410 comp rx=0 tx=0).stop 2024-09-21T03:11:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 -- 172.21.15.16:0/2350979654 shutdown_connections 2024-09-21T03:11:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 --2- 172.21.15.16:0/2350979654 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f17281086d0 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.440 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 --2- 172.21.15.16:0/2350979654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728108c10 0x7f172810f440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 --2- 172.21.15.16:0/2350979654 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728102990 0x7f1728102da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 -- 172.21.15.16:0/2350979654 >> 172.21.15.16:0/2350979654 conn(0x7f17280fe3b0 msgr2=0x7f1728100800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 -- 172.21.15.16:0/2350979654 shutdown_connections 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.438+0000 7f173024a640 1 -- 172.21.15.16:0/2350979654 wait complete. 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 Processor -- start 2024-09-21T03:11:32.441 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 -- start start 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 0x7f17281992d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728108c10 0x7f172819eb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f17281128d0 con 0x7f17281082c0 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f1728112750 con 0x7f1728108c10 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f173024a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f1728112a50 con 0x7f1728102990 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f172d7e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.439+0000 7f172d7e5640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40928/0 (socket says 172.21.15.16:40928) 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 -- 172.21.15.16:0/3485321286 learned_addr learned my addr 172.21.15.16:0/3485321286 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172dfe6640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 0x7f17281992d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.443 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172e7e7640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728108c10 0x7f172819eb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 -- 172.21.15.16:0/3485321286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 msgr2=0x7f17281992d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 0x7f17281992d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 -- 172.21.15.16:0/3485321286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728108c10 msgr2=0x7f172819eb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728108c10 0x7f172819eb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 -- 172.21.15.16:0/3485321286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f172819f200 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172dfe6640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 0x7f17281992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f172d7e5640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f171c02a930 tx=0x7f171c031a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c03f030 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f172819f490 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f171c004490 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f171c035070 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.440+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17281a5c80 con 0x7f17281082c0 2024-09-21T03:11:32.444 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.442+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1728102da0 con 0x7f17281082c0 2024-09-21T03:11:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.443+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f171c042030 con 0x7f17281082c0 2024-09-21T03:11:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.444+0000 7f1726ffd640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 0x7f17100b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.444+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f171c107f20 con 0x7f17281082c0 2024-09-21T03:11:32.445 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.444+0000 7f172dfe6640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 0x7f17100b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.446 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.444+0000 7f172dfe6640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 0x7f17100b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1718005fd0 tx=0x7f1718009500 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.448 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.447+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f171c0ce460 con 0x7f17281082c0 2024-09-21T03:11:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.637+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f172810aa30 con 0x7f17281082c0 2024-09-21T03:11:32.639 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.638+0000 7f1726ffd640 1 -- 172.21.15.16:0/3485321286 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f171c0d2840 con 0x7f17281082c0 2024-09-21T03:11:32.640 INFO:teuthology.orchestra.run.smithi016.stdout:pg_num: 64 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 msgr2=0x7f17100b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 0x7f17100b8b90 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f1718005fd0 tx=0x7f1718009500 comp rx=0 tx=0).stop 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 msgr2=0x7f1728199870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f171c02a930 tx=0x7f171c031a20 comp rx=0 tx=0).stop 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 shutdown_connections 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f17100b6680 0x7f17100b8b90 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f17281082c0 0x7f1728199870 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.642 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f1728108c10 0x7f172819eb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 --2- 172.21.15.16:0/3485321286 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1728102990 0x7f17281992d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 >> 172.21.15.16:0/3485321286 conn(0x7f17280fe3b0 msgr2=0x7f17281007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 shutdown_connections 2024-09-21T03:11:32.643 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.640+0000 7f173024a640 1 -- 172.21.15.16:0/3485321286 wait complete. 2024-09-21T03:11:32.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:32 smithi016 ceph-mon[31253]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:11:32.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:32 smithi016 ceph-mon[31253]: pgmap v666: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 123 op/s 2024-09-21T03:11:32.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:32 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1120561603' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:11:32.652 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:32 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2159356816' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:11:32.653 DEBUG:teuthology.orchestra.run.smithi016:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-09-21T03:11:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:32 smithi018 ceph-mon[34587]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:11:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:32 smithi018 ceph-mon[34587]: pgmap v666: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 123 op/s 2024-09-21T03:11:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:32 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1120561603' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:11:32.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:32 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2159356816' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:11:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:32 smithi123 ceph-mon[34645]: osdmap e89: 12 total, 12 up, 12 in 2024-09-21T03:11:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:32 smithi123 ceph-mon[34645]: pgmap v666: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 123 op/s 2024-09-21T03:11:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:32 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1120561603' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:11:32.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:32 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2159356816' entity='client.admin' cmd={"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} : dispatch 2024-09-21T03:11:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.763+0000 7f579f989640 1 -- 172.21.15.16:0/936871456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 msgr2=0x7f579810f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.763+0000 7f579f989640 1 --2- 172.21.15.16:0/936871456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579810f430 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f5790009ca0 tx=0x7f579002a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.765 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 -- 172.21.15.16:0/936871456 shutdown_connections 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 --2- 172.21.15.16:0/936871456 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579810f430 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 --2- 172.21.15.16:0/936871456 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f57981082b0 0x7f57981086c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 --2- 172.21.15.16:0/936871456 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f5798102980 0x7f5798102d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 -- 172.21.15.16:0/936871456 >> 172.21.15.16:0/936871456 conn(0x7f57980fe380 msgr2=0x7f57981007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 -- 172.21.15.16:0/936871456 shutdown_connections 2024-09-21T03:11:32.766 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 -- 172.21.15.16:0/936871456 wait complete. 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 Processor -- start 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.764+0000 7f579f989640 1 -- start start 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102980 0x7f579819a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57981082b0 0x7f579819a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f57981139e0 con 0x7f5798108c00 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f5798113860 con 0x7f57981082b0 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579f989640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f5798113b60 con 0x7f5798102980 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:40936/0 (socket says 172.21.15.16:40936) 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 -- 172.21.15.16:0/3839132507 learned_addr learned my addr 172.21.15.16:0/3839132507 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:32.768 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579d725640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102980 0x7f579819a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579cf24640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57981082b0 0x7f579819a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 -- 172.21.15.16:0/3839132507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102980 msgr2=0x7f579819a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102980 0x7f579819a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 -- 172.21.15.16:0/3839132507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57981082b0 msgr2=0x7f579819a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57981082b0 0x7f579819a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 -- 172.21.15.16:0/3839132507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57981a0290 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.765+0000 7f579df26640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f57900098d0 tx=0x7f57900317b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.766+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790003b70 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.766+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5790003d10 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.766+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579002fa20 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.766+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57981a0520 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.766+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a6d10 con 0x7f5798108c00 2024-09-21T03:11:32.769 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.767+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5798103bf0 con 0x7f5798108c00 2024-09-21T03:11:32.770 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.768+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f579002fbe0 con 0x7f5798108c00 2024-09-21T03:11:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.769+0000 7f578a7fc640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 0x7f57840b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.770+0000 7f579d725640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 0x7f57840b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.770+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f5790108ad0 con 0x7f5798108c00 2024-09-21T03:11:32.771 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.770+0000 7f579d725640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 0x7f57840b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f578c009bd0 tx=0x7f578c005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:32.774 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:32.773+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f57900cf010 con 0x7f5798108c00 2024-09-21T03:11:33.019 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.017+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f5798102d90 con 0x7f5798108c00 2024-09-21T03:11:33.027 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.026+0000 7f578a7fc640 1 -- 172.21.15.16:0/3839132507 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v24) ==== 81+0+834054 (secure 0 0 0) 0x7f57900d33f0 con 0x7f5798108c00 2024-09-21T03:11:33.037 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:33.037 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:33.037 INFO:teuthology.orchestra.run.smithi016.stdout: "epoch": 24, 2024-09-21T03:11:33.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.037 INFO:teuthology.orchestra.run.smithi016.stdout: "active_gid": 14226, 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "active_name": "x", 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addrs": { 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6800", 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3994309618 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v1", 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:6801", 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3994309618 2024-09-21T03:11:33.038 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "active_addr": "172.21.15.16:6801/3994309618", 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "active_change": "2024-09-21T02:51:53.053676+0000", 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "active_mgr_features": 4540719139924082687, 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "available": true, 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "standbys": [ 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24136, 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "z", 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:11:33.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.041 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.048 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:11:33.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:11:33.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:11:33.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:11:33.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:11:33.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:11:33.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:11:33.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.064 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.067 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.069 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:11:33.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.073 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.075 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.076 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:11:33.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:11:33.083 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:11:33.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:11:33.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.105 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:11:33.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:11:33.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:11:33.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.120 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:11:33.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:11:33.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:11:33.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:11:33.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:11:33.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:33.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.131 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.136 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.137 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.138 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.141 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.142 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:11:33.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.144 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.145 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.156 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.169 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.173 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:11:33.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:11:33.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:11:33.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.183 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:11:33.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.194 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.200 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.203 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.204 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.208 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.212 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:33.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:33.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:11:33.218 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.219 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.220 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:11:33.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:11:33.225 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.228 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.234 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.235 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.236 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:11:33.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:11:33.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:11:33.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.250 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.251 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:33.252 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:11:33.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:11:33.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.257 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.259 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.260 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.262 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.264 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.266 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.267 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.270 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.272 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.273 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.275 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.285 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.286 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.287 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.290 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.296 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.298 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:11:33.301 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.302 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.306 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.308 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.309 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.312 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.314 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:11:33.316 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.317 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.321 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.324 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.327 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.328 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.329 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.333 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:11:33.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.338 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:11:33.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.341 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.347 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.350 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.354 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.355 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.360 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:11:33.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.366 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.367 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.369 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:11:33.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:11:33.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.378 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.387 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.388 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.391 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:11:33.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.394 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.395 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.396 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.403 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:11:33.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.419 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.420 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.424 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.425 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.426 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.427 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.428 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.429 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.430 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.431 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.432 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.433 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.434 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.435 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.436 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.437 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.438 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.439 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.440 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.441 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.442 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.443 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.444 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.445 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.446 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.447 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:11:33.448 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.449 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.450 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.451 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.452 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.453 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.454 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.455 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:11:33.456 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:11:33.459 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.460 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.461 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.462 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.463 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.464 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:11:33.465 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.466 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:11:33.467 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:11:33.468 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.469 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.470 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.471 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.472 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.473 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.474 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.475 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.476 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:11:33.477 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.478 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.479 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.480 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.481 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.482 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.483 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.484 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.485 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.486 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.487 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.488 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.489 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.490 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:11:33.491 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.492 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.493 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:11:33.494 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.495 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.496 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.497 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.498 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.499 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.500 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.501 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.502 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:11:33.503 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:11:33.504 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.505 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.506 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "gid": 24170, 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "y", 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "mgr_features": 4540719139924082687, 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.507 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.508 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.509 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.510 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.511 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.512 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.513 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.514 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.515 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.516 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:11:33.517 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:11:33.518 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:11:33.519 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.520 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.521 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.522 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.523 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.524 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.525 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:11:33.526 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.527 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:11:33.528 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.529 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:11:33.530 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.531 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.532 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.533 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.534 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.535 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.536 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:11:33.537 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.538 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.539 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.540 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.541 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.542 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:11:33.543 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.544 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:11:33.545 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:11:33.546 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.547 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.548 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.549 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:11:33.550 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.551 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.552 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.553 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.554 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:11:33.555 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.556 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:11:33.557 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.558 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.559 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.560 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.561 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.562 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.563 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.564 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.565 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:11:33.566 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:11:33.567 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.568 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:11:33.569 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.570 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.571 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.572 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.573 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.574 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.575 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.576 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.577 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:11:33.578 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:11:33.579 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:11:33.580 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.581 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.582 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.583 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.584 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.585 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.586 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.587 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.588 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.589 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:11:33.590 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.591 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.592 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:11:33.593 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:11:33.594 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:11:33.595 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.596 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.597 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.598 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.599 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.600 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.601 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.602 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.603 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.604 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.605 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.606 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.607 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.608 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:11:33.609 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.610 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.611 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.612 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.613 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.614 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.615 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.616 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.617 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.618 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.619 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.620 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.621 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:11:33.622 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.623 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.624 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.625 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.626 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.627 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.628 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.629 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:11:33.630 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:11:33.631 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.632 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.633 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.634 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.635 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.636 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.637 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.638 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.639 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.640 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.641 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.642 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.643 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.644 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.645 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.646 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:11:33.647 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:11:33.648 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.649 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.650 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.651 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.652 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.653 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.654 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.655 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.656 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.657 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.658 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.659 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.660 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:11:33.661 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.662 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.663 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.664 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.665 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.666 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.667 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.668 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.669 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.670 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.671 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.672 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.673 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.674 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.675 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.676 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.677 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.678 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.679 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.680 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:11:33.681 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.682 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.683 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.684 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.685 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.686 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.687 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.688 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.689 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.690 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.691 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.692 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.693 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.694 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.695 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.696 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.696 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.696 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.697 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.697 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.697 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.698 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.699 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.700 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.701 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.702 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:33.703 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.704 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.705 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.706 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.707 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.708 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.709 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.710 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.711 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.712 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:33.713 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:11:33.714 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.715 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.716 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:11:33.717 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:11:33.718 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:11:33.719 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.720 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.721 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.722 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.723 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.724 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.725 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.726 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:11:33.727 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.728 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:33.729 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:11:33.730 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:11:33.731 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:33.732 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.733 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.734 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.735 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.736 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.737 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.738 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.739 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.740 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.741 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.742 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.743 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.744 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.745 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.746 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.747 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.748 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.749 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.750 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.751 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.752 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.753 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.754 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.755 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.756 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.757 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.758 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.759 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.760 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.761 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:11:33.762 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.763 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.764 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.765 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.766 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.767 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.768 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.769 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.770 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.771 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.772 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.773 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.774 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.775 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.776 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.777 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.778 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.779 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.780 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.781 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.782 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.783 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.784 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.785 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.786 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.787 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.788 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:11:33.789 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.790 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.791 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.792 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.793 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.794 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.795 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.796 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.797 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.798 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.799 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.800 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.801 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.802 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.803 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.804 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.805 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.806 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.807 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.808 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.809 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.810 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.811 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.812 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.813 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.814 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.815 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.816 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.817 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.818 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.819 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.820 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:11:33.821 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:11:33.822 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.823 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.824 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.825 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.826 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.827 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.828 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.829 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.830 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.831 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.832 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:11:33.833 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:33.834 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:33.835 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.836 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.837 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.838 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.839 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.840 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.841 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.842 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.843 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.844 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:11:33.845 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:33.846 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.847 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.848 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.849 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.850 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.851 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:11:33.852 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.853 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.854 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.855 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.856 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.857 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.858 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.859 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.860 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.861 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.862 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.863 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.864 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.865 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.866 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.867 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.868 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.869 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.870 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.871 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.872 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.873 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.874 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.875 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.876 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.877 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:11:33.878 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.879 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.880 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.881 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.882 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.883 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.884 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.885 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.886 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.887 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.888 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.889 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.890 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.891 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.892 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.893 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.894 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.895 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.896 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.897 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.898 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.899 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.900 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.901 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.902 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.903 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.904 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.905 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.906 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.907 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.908 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.909 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.910 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.911 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.912 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.913 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.914 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.915 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.916 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.917 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.918 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.919 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.920 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.921 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.922 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.923 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:11:33.924 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.925 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.926 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.927 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.928 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.929 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.930 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.931 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.932 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.933 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.934 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.935 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.936 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.937 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.938 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.939 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.940 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:11:33.941 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.942 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.943 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.944 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.945 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.946 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.947 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.948 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.949 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.950 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.951 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.952 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.953 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.954 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.955 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.956 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.957 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.958 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.959 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.960 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.961 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.962 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.963 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.964 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.965 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.966 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.967 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:11:33.968 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.969 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.970 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:33.971 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "modules": [ 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "iostat", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "nfs", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "restful", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "snap_schedule" 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "available_modules": [ 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "alerts", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.972 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How frequently to reexamine health status", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.973 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.974 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.975 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.976 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.977 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_destination": { 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_destination", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.978 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email address to send alerts to, use commas to separate multiple", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_from_name": { 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_from_name", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "Ceph", 2024-09-21T03:11:33.979 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Email From: name", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_host": { 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_host", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.980 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP server", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_password": { 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_password", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.981 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Password to authenticate with", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_port": { 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_port", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "465", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.982 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP port", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_sender": { 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_sender", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SMTP envelope sender", 2024-09-21T03:11:33.983 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_ssl": { 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_ssl", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use SSL to connect to SMTP server", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.984 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "smtp_user": { 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smtp_user", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "User to authenticate as", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.985 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.986 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "balancer", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "active": { 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "active", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.987 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically balance PGs across cluster", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_time": { 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_time", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0000", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.988 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "beginning time of day to automatically balance", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "begin_weekday": { 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "begin_weekday", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.989 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_max_iterations": { 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_max_iterations", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "25", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "250", 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.990 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_metrics": { 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_metrics", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "pgs,objects,bytes", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.991 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "crush_compat_step": { 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crush_compat_step", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.5", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0.001", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "0.999", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "aggressiveness of optimization", 2024-09-21T03:11:33.992 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "end_time": { 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_time", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2359", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ending time of day to automatically balance", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.993 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "end_weekday": { 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "end_weekday", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "0", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "6", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.994 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.995 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.996 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:33.997 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.998 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "min_score": { 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_score", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:33.999 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "upmap", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.000 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "crush-compat", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "none", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "read", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap-read" 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Balancer mode", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_ids": { 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_ids", 2024-09-21T03:11:34.001 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:34.002 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.003 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_deviation": { 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_deviation", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.004 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "deviation below which no optimization is attempted", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "upmap_max_optimizations": { 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "upmap_max_optimizations", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.005 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.006 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_down_multiplier": { 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_down_multiplier", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.007 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_refresh_rate": { 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_refresh_rate", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "20", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "agent_starting_port": { 2024-09-21T03:11:34.008 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "agent_starting_port", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4721", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_ptrace": { 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_ptrace", 2024-09-21T03:11:34.009 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_interval": { 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_interval", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.010 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to autotune daemon memory", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "autotune_memory_target_ratio": { 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "autotune_memory_target_ratio", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.011 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0.7", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm_log_destination": { 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cephadm_log_destination", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.012 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "file", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "file,syslog", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "syslog" 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "cgroups_split": { 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cgroups_split", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.013 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "config_checks_enabled": { 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_checks_enabled", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.014 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "config_dashboard": { 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_dashboard", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.015 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_alertmanager": { 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_alertmanager", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.27.0", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.016 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_base": { 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_base", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/ceph", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.017 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Container image name, without the tag", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_elasticsearch": { 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_elasticsearch", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.018 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "elasticsearch container image", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_grafana": { 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_grafana", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/grafana:10.4.8", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:34.019 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_haproxy": { 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_haproxy", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "HAproxy container image", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.020 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_agent": { 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_agent", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger agent container image", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.021 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_collector": { 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_collector", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger collector container image", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_jaeger_query": { 2024-09-21T03:11:34.022 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_jaeger_query", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Jaeger query container image", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_keepalived": { 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_keepalived", 2024-09-21T03:11:34.023 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Keepalived container image", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_loki": { 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_loki", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.024 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/loki:3.0.0", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Loki container image", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nginx": { 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nginx", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.025 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nginx:sclorg-nginx-126", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nginx container image", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_node_exporter": { 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_node_exporter", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.026 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.7.0", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_nvmeof": { 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_nvmeof", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.17", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.027 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Nvme-of container image", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_oauth2_proxy": { 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_oauth2_proxy", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.028 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/oauth2-proxy/oauth2-proxy:v7.6.0", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "oauth2-proxy container image", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_prometheus": { 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_prometheus", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.029 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/prometheus/prometheus:v2.51.0", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Prometheus container image", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_promtail": { 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_promtail", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/grafana/promtail:3.0.0", 2024-09-21T03:11:34.030 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Promtail container image", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba": { 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-09-21T03:11:34.031 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB container image", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_samba_metrics": { 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_samba_metrics", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "quay.io/samba.org/samba-metrics:latest", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.032 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Samba/SMB metrics exporter container image", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "container_image_snmp_gateway": { 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_image_snmp_gateway", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.033 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "SNMP Gateway container image", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "container_init": { 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "container_init", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.034 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Run podman/docker with `--init`", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "daemon_cache_timeout": { 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "daemon_cache_timeout", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.035 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_cephadm_command_timeout": { 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_cephadm_command_timeout", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "900", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-09-21T03:11:34.036 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_registry": { 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "default_registry", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "docker.io", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.037 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "device_cache_timeout": { 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_cache_timeout", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1800", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache device inventory", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.038 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "device_enhanced_scan": { 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_enhanced_scan", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use libstoragemgmt during device scans", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.039 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "facts_cache_timeout": { 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "facts_cache_timeout", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "seconds to cache host facts data", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "grafana_dashboards_path": { 2024-09-21T03:11:34.040 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "grafana_dashboards_path", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "host_check_interval": { 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "host_check_interval", 2024-09-21T03:11:34.041 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to perform a host check", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "hw_monitoring": { 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hw_monitoring", 2024-09-21T03:11:34.042 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "inventory_list_all": { 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "inventory_list_all", 2024-09-21T03:11:34.043 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.044 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.045 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "log_refresh_metadata": { 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_refresh_metadata", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.046 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.047 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.048 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.049 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf": { 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.050 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "*", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.051 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max_count_per_host": { 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_count_per_host", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.052 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of daemons per service per host", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max_osd_draining_count": { 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_osd_draining_count", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.053 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "migration_current": { 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "migration_current", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "internal - do not modify", 2024-09-21T03:11:34.054 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "mode": { 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mode", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "root", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm-package", 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: "root" 2024-09-21T03:11:34.055 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "mode for remote execution of cephadm", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "oob_default_addr": { 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "oob_default_addr", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "169.254.1.1", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.056 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Default address for RedFish API (oob management).", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "prometheus_alerts_path": { 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus_alerts_path", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-09-21T03:11:34.057 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_insecure": { 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_insecure", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-09-21T03:11:34.058 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_password": { 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_password", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-09-21T03:11:34.059 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_url": { 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_url", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.060 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "registry_username": { 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "registry_username", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.061 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "secure_monitoring_stack": { 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secure_monitoring_stack", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.062 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "service_discovery_port": { 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "service_discovery_port", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8765", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "cephadm service discovery port", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.063 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.064 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_config_file": { 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_config_file", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_count_max": { 2024-09-21T03:11:34.065 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_count_max", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many times ssh connections can fail liveness checks before the host is marked offline", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "ssh_keepalive_interval": { 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssh_keepalive_interval", 2024-09-21T03:11:34.066 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How often ssh connections are checked for liveness", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "use_agent": { 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_agent", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.067 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "use_repo_digest": { 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "use_repo_digest", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.068 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_failed_host_check": { 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_failed_host_check", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.069 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if the host check fails", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_daemons": { 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_daemons", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.070 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_on_stray_hosts": { 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_on_stray_hosts", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.071 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crash", 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.072 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.073 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.074 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.075 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.076 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "retain_interval": { 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retain_interval", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.077 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "31536000", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain crashes before pruning them", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.078 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_recent_interval": { 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_recent_interval", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1209600", 2024-09-21T03:11:34.079 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "time interval in which to warn about recent crashes", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dashboard", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-09-21T03:11:34.080 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_HOST": { 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_HOST", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.081 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.082 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_ENABLED": { 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_ENABLED", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.083 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.084 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "ENABLE_BROWSABLE_API": { 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ENABLE_BROWSABLE_API", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.085 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.086 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.087 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.088 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.089 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_NFS": { 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.090 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RBD": { 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.091 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "FEATURE_TOGGLE_RGW": { 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.092 INFO:teuthology.orchestra.run.smithi016.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_PASSWORD": { 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_PASSWORD", 2024-09-21T03:11:34.093 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.094 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_URL": { 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_URL", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.095 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_API_USERNAME": { 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_API_USERNAME", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.096 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.097 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.098 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.099 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "ISSUE_TRACKER_API_KEY": { 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.100 INFO:teuthology.orchestra.run.smithi016.stdout: "MANAGED_BY_CLUSTERS": { 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "[]", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "MULTICLUSTER_CONFIG": { 2024-09-21T03:11:34.101 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "MULTICLUSTER_CONFIG", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "{}", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_HOST": { 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_HOST", 2024-09-21T03:11:34.102 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.103 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.104 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.105 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.106 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.107 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.108 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.109 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.110 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.111 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_ENABLED": { 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_ENABLED", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.112 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-09-21T03:11:34.113 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.114 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "REST_REQUESTS_TIMEOUT": { 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.115 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "45", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ACCESS_KEY": { 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ACCESS_KEY", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.116 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "admin", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.117 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SECRET_KEY": { 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SECRET_KEY", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.118 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "RGW_API_SSL_VERIFY": { 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "RGW_API_SSL_VERIFY", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.119 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "UNSAFE_TLS_v1_2": { 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "UNSAFE_TLS_v1_2", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.120 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.121 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-09-21T03:11:34.122 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "cross_origin_url": { 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cross_origin_url", 2024-09-21T03:11:34.123 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "crt_file": { 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "crt_file", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.124 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "debug": { 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "debug", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.125 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Enable/disable debug options", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "jwt_token_ttl": { 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "jwt_token_ttl", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.126 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "28800", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.127 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.128 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.129 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.130 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.131 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.132 INFO:teuthology.orchestra.run.smithi016.stdout: "motd": { 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "motd", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "The message of the day", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect_resolve_ip_addr": { 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "redirect_resolve_ip_addr", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.133 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:34.134 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:34.135 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8080", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.136 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.137 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl_server_port": { 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl_server_port", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8443", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.138 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "sso_oauth2": { 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sso_oauth2", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.139 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "redirect", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "redirect" 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.140 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.141 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "url_prefix": { 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url_prefix", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.142 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_monitoring": { 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_monitoring", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.143 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "monitor device health metrics", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.144 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.145 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.146 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.147 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.148 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "mark_out_threshold": { 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mark_out_threshold", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "2419200", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.149 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "pool_name": { 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pool_name", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "device_health_metrics", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name of pool in which to store device health metrics", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.150 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "retention_period": { 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "retention_period", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15552000", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long to retain device health metrics", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.151 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_frequency": { 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_frequency", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to scrape device health metrics", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "self_heal": { 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "self_heal", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.152 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:34.153 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how frequently to wake up and check device health", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.154 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "warn_threshold": { 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "warn_threshold", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7257600", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.155 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "diskprediction_local", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.156 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.157 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.158 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.159 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.160 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "predict_interval": { 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predict_interval", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "86400", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.161 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "predictor_model": { 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "predictor_model", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "prophetstor", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.162 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "600", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.163 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.164 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "influx", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": false, 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "influxdb python module not found", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "batch_size": { 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "batch_size", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5000", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.165 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "database": { 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "database", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "ceph", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.166 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "hostname": { 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "hostname", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server hostname", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.167 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "30", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "5", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.168 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.169 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.170 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.171 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.172 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "password": { 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "password", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "password of InfluxDB server user", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.173 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "port": { 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "port", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "8086", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "InfluxDB server port", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.174 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.175 INFO:teuthology.orchestra.run.smithi016.stdout: "ssl": { 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ssl", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "false", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.176 INFO:teuthology.orchestra.run.smithi016.stdout: "threads": { 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threads", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "32", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "username": { 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "username", 2024-09-21T03:11:34.177 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "username of InfluxDB server user", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "verify_ssl": { 2024-09-21T03:11:34.178 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "verify_ssl", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "true", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.179 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "insights", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.180 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.181 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.182 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.183 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.184 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "iostat", 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.185 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.186 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.187 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.188 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.189 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.190 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "k8sevents", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.191 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "ceph_event_retention_days": { 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "ceph_event_retention_days", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "7", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Days to hold ceph event information within local cache", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.192 INFO:teuthology.orchestra.run.smithi016.stdout: "config_check_secs": { 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "config_check_secs", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "10", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.193 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.194 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.195 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.196 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.197 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.198 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.199 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "localpool", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "failure_domain": { 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "failure_domain", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "host", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "failure domain for any created local pool", 2024-09-21T03:11:34.200 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.201 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.202 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.203 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.204 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.205 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min_size": { 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "min_size", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default min_size for any created local pool", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.206 INFO:teuthology.orchestra.run.smithi016.stdout: "num_rep": { 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "num_rep", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default replica count for any created local pool", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_num": { 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_num", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "128", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "default pg_num for any created local pool", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.208 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "prefix": { 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prefix", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "name prefix for any created local pool", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.209 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.210 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "subtree": { 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "subtree", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "rack", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.211 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "CRUSH level for which to create a local pool", 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mds_autoscaler", 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.212 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.213 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.214 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.215 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.216 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.217 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirroring", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.218 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.219 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.220 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.221 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.222 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.223 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "nfs", 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.224 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.225 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.226 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.227 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.228 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.229 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.230 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "fail_fs": { 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "fail_fs", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.231 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.232 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.233 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.234 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.235 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator": { 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "orchestrator", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "cephadm", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "rook", 2024-09-21T03:11:34.236 INFO:teuthology.orchestra.run.smithi016.stdout: "test_orchestrator" 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Orchestrator backend", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.237 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_perf_query", 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.238 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.239 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.240 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.241 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.242 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.243 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "osd_support", 2024-09-21T03:11:34.244 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.245 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.246 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.247 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.248 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.249 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.250 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "pg_autoscaler", 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.251 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.252 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.253 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.254 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.255 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:34.256 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.257 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "threshold": { 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "threshold", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "3.0", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1.0", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.258 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "scaling threshold", 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "progress", 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_pg_recovery_event": { 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_pg_recovery_event", 2024-09-21T03:11:34.259 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow the module to show pg recovery progress", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:34.260 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.261 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.262 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.263 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.264 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.265 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "max_completed_events": { 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_completed_events", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.266 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "50", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "number of past completed events to remember", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "sleep_interval": { 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sleep_interval", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.267 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "how long the module is going to sleep", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.268 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.269 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "prometheus", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "cache": { 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "cache", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.270 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "exclude_perf_counters": { 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "exclude_perf_counters", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-09-21T03:11:34.271 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.272 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.273 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.274 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.275 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.276 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools": { 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.277 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_stats_pools_refresh_interval": { 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "300", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.278 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "scrape_interval": { 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "scrape_interval", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15.0", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.279 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "::", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.280 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "9283", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-09-21T03:11:34.281 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.282 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "stale_cache_strategy": { 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stale_cache_strategy", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "log", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.283 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_behaviour": { 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_behaviour", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "default", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "default", 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: "error" 2024-09-21T03:11:34.284 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "standby_error_status_code": { 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "standby_error_status_code", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "400", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "599", 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.285 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.286 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.287 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.288 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.289 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.290 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_snap_create": { 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_snap_create", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.291 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "mirror_snapshot_schedule": { 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "mirror_snapshot_schedule", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.292 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.293 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "trash_purge_schedule": { 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "trash_purge_schedule", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.294 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "restful", 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "enable_auth": { 2024-09-21T03:11:34.295 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enable_auth", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "key_file": { 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "key_file", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.296 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.297 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.298 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.299 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.300 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.301 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max_requests": { 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_requests", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "500", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Maximum number of requests to keep in memory. When new request comes in, the oldest request will be removed if the number of requests exceeds the max request number.if un-finished request is removed, error message will be logged in the ceph-mgr log.", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.302 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "server_addr": { 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_addr", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "server_port": { 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "server_port", 2024-09-21T03:11:34.303 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.304 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rgw", 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.305 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.306 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.307 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.308 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.309 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.310 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "secondary_zone_period_retry_limit": { 2024-09-21T03:11:34.311 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "secondary_zone_period_retry_limit", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "5", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "RGW module period update retry limit for secondary site", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.312 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rook", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.313 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.314 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.315 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.316 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.317 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.318 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "storage_class": { 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "storage_class", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "local", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.319 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "selftest", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.320 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.321 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.322 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.323 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.324 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "roption1": { 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption1", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.325 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "roption2": { 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "roption2", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "xyz", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.326 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption1": { 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption1", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.327 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption2": { 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption2", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.328 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption3": { 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption3", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "float", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.329 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption4": { 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption4", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption5": { 2024-09-21T03:11:34.330 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption5", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption6": { 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption6", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.331 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "rwoption7": { 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rwoption7", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "1", 2024-09-21T03:11:34.332 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "42", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.333 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "testkey": { 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testkey", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.334 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "testlkey": { 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testlkey", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "testnewline": { 2024-09-21T03:11:34.335 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "testnewline", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.336 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "smb", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "internal_store_backend": { 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "internal_store_backend", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "set internal store backend. for develoment and testing only", 2024-09-21T03:11:34.337 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.338 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.339 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.340 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.341 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.342 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "update_orchestration": { 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "update_orchestration", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.343 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "automatically update orchestration when smb resources are changed", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.344 INFO:teuthology.orchestra.run.smithi016.stdout: "allow_m_granularity": { 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "allow_m_granularity", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "allow minute scheduled snapshots", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "dump_on_update": { 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "dump_on_update", 2024-09-21T03:11:34.345 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "dump database to debug log on update", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.346 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.347 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.348 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.349 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.350 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.351 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.352 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "stats", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.353 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.354 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.355 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.356 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.357 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.358 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "status", 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.359 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.360 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.361 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.362 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.363 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.364 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.365 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telegraf", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "address": { 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "address", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-09-21T03:11:34.366 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "15", 2024-09-21T03:11:34.367 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.368 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.369 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.370 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.371 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.372 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.373 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "telemetry", 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.374 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_basic": { 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_basic", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share basic cluster information (size, version)", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.375 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_crash": { 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_crash", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.376 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_device": { 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_device", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_ident": { 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_ident", 2024-09-21T03:11:34.377 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "channel_perf": { 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "channel_perf", 2024-09-21T03:11:34.378 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Share various performance metrics of a cluster", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "contact": { 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "contact", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.379 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "description": { 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "description", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.380 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "device_url": { 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "device_url", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.381 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "enabled": { 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "enabled", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.382 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.383 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "24", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "8", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "last_opt_revision": { 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "last_opt_revision", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.384 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "1", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard": { 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.385 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "leaderboard_description": { 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "leaderboard_description", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.386 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.387 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.388 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.389 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.390 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.391 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "organization": { 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "organization", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.392 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "proxy": { 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "proxy", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.393 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.394 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "url": { 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "url", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.395 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "test_orchestrator", 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.396 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.397 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.398 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.399 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.400 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.401 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.402 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.403 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.404 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.405 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.406 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.407 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max_concurrent_clones": { 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "max_concurrent_clones", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "4", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Number of asynchronous cloner threads", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.408 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "periodic_async_work": { 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "periodic_async_work", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Periodically check for async work", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.409 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_delay": { 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_delay", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.410 INFO:teuthology.orchestra.run.smithi016.stdout: "snapshot_clone_no_wait": { 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snapshot_clone_no_wait", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "True", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.411 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.412 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "can_run": true, 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "error_string": "", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "module_options": { 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "discovery_interval": { 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "discovery_interval", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "uint", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "100", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.413 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "identifier": { 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "identifier", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.414 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "interval": { 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "interval", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "secs", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "60", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.415 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "log_level": { 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_level", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.416 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster": { 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.417 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_cluster_level": { 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_cluster_level", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "info", 2024-09-21T03:11:34.418 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [ 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "critical", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "debug", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "error", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "info", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "warning" 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.419 INFO:teuthology.orchestra.run.smithi016.stdout: "log_to_file": { 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "log_to_file", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "bool", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "False", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "sqlite3_killpoint": { 2024-09-21T03:11:34.420 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "sqlite3_killpoint", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "dev", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 1, 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "0", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_host": { 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_host", 2024-09-21T03:11:34.421 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_port": { 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_port", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "int", 2024-09-21T03:11:34.422 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "10051", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "zabbix_sender": { 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "zabbix_sender", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "str", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "level": "advanced", 2024-09-21T03:11:34.423 INFO:teuthology.orchestra.run.smithi016.stdout: "flags": 0, 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "min": "", 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "max": "", 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "enum_allowed": [], 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "desc": "", 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "long_desc": "", 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "tags": [], 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "see_also": [] 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "services": { 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: "dashboard": "https://172.21.15.16:8443/" 2024-09-21T03:11:34.424 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "always_on_modules": { 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "octopus": [ 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "pacific": [ 2024-09-21T03:11:34.425 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "quincy": [ 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:11:34.426 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "reef": [ 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:11:34.427 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: ], 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "squid": [ 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "balancer", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "crash", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "devicehealth", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "orchestrator", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "pg_autoscaler", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "progress", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "rbd_support", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "status", 2024-09-21T03:11:34.428 INFO:teuthology.orchestra.run.smithi016.stdout: "telemetry", 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "volumes" 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "last_failure_osd_epoch": 5, 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "active_clients": [ 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "devicehealth", 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 2855103234 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.429 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "libcephsqlite", 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 3468954010 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "rbd_support", 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.430 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1946590608 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1556948220 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.431 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "snap_schedule", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1775037903 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "name": "volumes", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addrvec": [ 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: { 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "type": "v2", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "addr": "172.21.15.16:0", 2024-09-21T03:11:34.432 INFO:teuthology.orchestra.run.smithi016.stdout: "nonce": 1874751702 2024-09-21T03:11:34.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.433 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.433 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:34.433 INFO:teuthology.orchestra.run.smithi016.stdout: ] 2024-09-21T03:11:34.433 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/1804946991' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3485321286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:33 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3839132507' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/1804946991' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3485321286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.434 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:33 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3839132507' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:11:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 msgr2=0x7f57840b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 0x7f57840b8c10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f578c009bd0 tx=0x7f578c005f90 comp rx=0 tx=0).stop 2024-09-21T03:11:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 msgr2=0x7f579819fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:34.435 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f57900098d0 tx=0x7f57900317b0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 shutdown_connections 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f57840b6700 0x7f57840b8c10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f5798108c00 0x7f579819fb90 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f57981082b0 0x7f579819a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 --2- 172.21.15.16:0/3839132507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5798102980 0x7f579819a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 >> 172.21.15.16:0/3839132507 conn(0x7f57980fe380 msgr2=0x7f5798105800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 shutdown_connections 2024-09-21T03:11:34.436 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:33.032+0000 7f579f989640 1 -- 172.21.15.16:0/3839132507 wait complete. 2024-09-21T03:11:34.482 DEBUG:tasks.check_counter:Getting stats from a 2024-09-21T03:11:34.482 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.a perf dump 2024-09-21T03:11:34.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/1804946991' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3485321286' entity='client.admin' cmd={"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} : dispatch 2024-09-21T03:11:34.483 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:33 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3839132507' entity='client.admin' cmd={"prefix": "mgr dump", "format": "json-pretty"} : dispatch 2024-09-21T03:11:34.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:34 smithi018 ceph-mon[34587]: pgmap v667: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 153 op/s 2024-09-21T03:11:34.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:34 smithi123 ceph-mon[34645]: pgmap v667: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 153 op/s 2024-09-21T03:11:34.754 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:34.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:34 smithi016 ceph-mon[31253]: pgmap v667: 97 pgs: 97 active+clean; 40 GiB data, 125 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 153 op/s 2024-09-21T03:11:35.455 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:35.455 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:35.455 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-21T03:11:35.455 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-21T03:11:35.455 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.003683890, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000053808, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002389751, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004090371, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001363457 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:35.456 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 163, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 108880, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:35.457 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.018073049, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013969651, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001923571, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 161, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.011016755, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000068427 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:35.458 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 108880 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 220, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 215, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 837472, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 132384, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.042546698, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.019974908, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.029381476, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:35.459 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 214, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.021006537, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000098161 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 837472, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 132384 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:11:35.460 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:35.461 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:35.462 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 148984, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:35.463 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:11:35.464 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 221, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 806372, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 221, 2024-09-21T03:11:35.465 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 221, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 806372, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:35.466 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:36.286 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.a 2024-09-21T03:11:36.286 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.a 2024-09-21T03:11:36.286 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.a 2024-09-21T03:11:36.286 DEBUG:tasks.check_counter:Getting stats from d 2024-09-21T03:11:36.286 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.d perf dump 2024-09-21T03:11:36.547 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:36.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:36 smithi018 ceph-mon[34587]: pgmap v668: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 260 op/s 2024-09-21T03:11:36.956 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:36 smithi016 ceph-mon[31253]: pgmap v668: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 260 op/s 2024-09-21T03:11:36.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:36 smithi123 ceph-mon[34645]: pgmap v668: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 260 op/s 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:37.079 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.001646419, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000417657, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.080 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 5, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 169, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 3104, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 110032, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.019208500, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.013962410, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.003482656, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:37.081 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 168, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.021929660, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000130533 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 110032 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 226, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 221, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 839168, 2024-09-21T03:11:37.082 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 135744, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043381823, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.020239735, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.031589521, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 220, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.020748211, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000094310 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:37.083 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 839168, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 135744 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:37.084 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:37.085 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:37.086 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149007, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:11:37.087 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:11:37.088 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 227, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 807216, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 227, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 227, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 807216, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:37.089 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:37.090 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:37.947 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-09-21T03:11:37.948 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:11:37.948 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-09-21T03:11:37.948 DEBUG:tasks.check_counter:Getting stats from g 2024-09-21T03:11:37.948 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.g perf dump 2024-09-21T03:11:38.212 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:38.721 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 225, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 218, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 840688, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 133472, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.049555635, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.020397600, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.035384415, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:38.722 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 218, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.042284551, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000193965 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 840688, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 133472 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 165, 2024-09-21T03:11:38.723 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 109232, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.017988112, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014258225, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001430191, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 163, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.035315847, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000216661 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:38.724 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 109232 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.000392921, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.000297596, 2024-09-21T03:11:38.725 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.726 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:38.727 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:38.728 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149047, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 31, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:38.729 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:11:38.730 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 222, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 806502, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 222, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 222, 2024-09-21T03:11:38.731 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 806502, 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:38.732 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:38.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:38 smithi018 ceph-mon[34587]: pgmap v669: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 264 op/s 2024-09-21T03:11:38.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:38 smithi123 ceph-mon[34645]: pgmap v669: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 264 op/s 2024-09-21T03:11:39.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:38 smithi016 ceph-mon[31253]: pgmap v669: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 264 op/s 2024-09-21T03:11:39.401 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.g 2024-09-21T03:11:39.401 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.g 2024-09-21T03:11:39.401 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.g 2024-09-21T03:11:39.401 DEBUG:tasks.check_counter:Getting stats from j 2024-09-21T03:11:39.401 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.j perf dump 2024-09-21T03:11:39.662 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout:{ 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 4, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 4, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 448, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.003323477, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.000049457, 2024-09-21T03:11:40.198 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.002289850, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 3, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.004084671, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.001361557 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:40.199 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 227, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 222, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 839440, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 136384, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.043875985, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.021279464, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.031593892, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 221, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.036324548, 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000164364 2024-09-21T03:11:40.200 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 839440, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 136384 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages": 169, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_bytes": 109936, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:40.201 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_total_time": 0.018352288, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_send_time": 0.014836782, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_recv_time": 0.001303667, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 167, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.016462821, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000098579 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.202 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "msgr_send_encrypted_bytes": 109936 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "cct": { 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "total_workers": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "unhealthy_workers": 0 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "mempool": { 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:40.203 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:40.204 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:40.205 INFO:teuthology.orchestra.run.smithi016.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_bytes": 149007, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_anon_items": 30, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_items": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:40.206 INFO:teuthology.orchestra.run.smithi016.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "pgmap_items": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "mds_co_items": 2, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_1_items": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "unittest_2_items": 0 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: }, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "val": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "max": 104857600, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_started": 0, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get": 228, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_sum": 807346, 2024-09-21T03:11:40.207 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "get_or_fail_success": 228, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "take": 0, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "take_sum": 0, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "put": 228, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "put_sum": 807346, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "wait": { 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgcount": 0, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "sum": 0.000000000, 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: "avgtime": 0.000000000 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout: } 2024-09-21T03:11:40.208 INFO:teuthology.orchestra.run.smithi016.stdout:} 2024-09-21T03:11:40.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:40 smithi018 ceph-mon[34587]: pgmap v670: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 231 op/s 2024-09-21T03:11:40.952 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:40 smithi016 ceph-mon[31253]: pgmap v670: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 231 op/s 2024-09-21T03:11:40.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:40 smithi123 ceph-mon[34645]: pgmap v670: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 17 KiB/s rd, 11 KiB/s wr, 231 op/s 2024-09-21T03:11:40.987 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-09-21T03:11:40.987 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:11:40.987 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-09-21T03:11:40.987 DEBUG:tasks.check_counter:Getting stats from b 2024-09-21T03:11:40.988 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.b perf dump 2024-09-21T03:11:41.256 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 4745, 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 4745, 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1241179, 2024-09-21T03:11:41.760 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 297266418, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 11, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 4, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 1.398601200, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.976647014, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.095600882, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.280747802, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4744, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.821988556, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000173269 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000184, 2024-09-21T03:11:41.761 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000092 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 4952, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 4507, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 1417494, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 295035094, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 24, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 1.385876294, 2024-09-21T03:11:41.762 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.953410546, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.120059222, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.285971891, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4505, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.964039890, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000213993 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 935, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000201596, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000215 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 338064, 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 481952 2024-09-21T03:11:41.763 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 202164, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 204331, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 81307575, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 188293795, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 137, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 14.538913543, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 7.756203226, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 16.943167213, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.199975258, 2024-09-21T03:11:41.764 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 204330, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 16.644252290, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000081457 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 325169, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.071816688, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000220 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 80732272, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 80889664 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-09-21T03:11:41.765 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 1, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-MDSRank": { 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 6063, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 65.226539523, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.010758129 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-PurgeQueue": { 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-09-21T03:11:41.766 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2513, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.085734279, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000034116 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "request": 271, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "reply": 271, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "reply_latency": { 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 271, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 30.713031016, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.113332217 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_reply": 0, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "forward": 0, 2024-09-21T03:11:41.767 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_complete": 0, 2024-09-21T03:11:41.768 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_keys": 0, 2024-09-21T03:11:41.768 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_commit": 23, 2024-09-21T03:11:41.768 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_split": 0, 2024-09-21T03:11:41.768 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_merge": 0, 2024-09-21T03:11:41.768 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes": 146, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_top": 51, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_bottom": 95, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pin_tail": 0, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pinned": 73, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_expired": 0, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_with_caps": 3, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "caps": 4, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "subtrees": 2, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse": 749, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_hit": 743, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_forward": 0, 2024-09-21T03:11:41.769 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_discover": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_remote_ino": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_lock": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "load_cent": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "q": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "exported": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "exported_inodes": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "imported": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "imported_inodes": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_dir_fetch": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_backtrace_fetch": 563, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_peer_discover": 0, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rfiles": 4, 2024-09-21T03:11:41.770 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rbytes": 234, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rsnaps": 6, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_fetch": 563, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_set_tag": 0, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_inodes": 199, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_base_inodes": 19, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dirfrag_rstats": 180, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_file_inodes": 364, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_revoke": 436, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_grant": 2803, 2024-09-21T03:11:41.771 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps": 198603, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps_dirty": 198221, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_cap_release": 2, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "process_request_cap_release": 47 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache": { 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update_receipt": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_try_discover": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_send_discover": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_handle_discover": 0, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays": 4, 2024-09-21T03:11:41.772 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_delayed": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_created": 22, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_enqueued": 18, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_reintegrated": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_migrated": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_processing": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_started": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_completed": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_enqueue_scrub": 19, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_exportdir": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_flush": 0, 2024-09-21T03:11:41.773 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragstats": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_inodestats": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_started": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_succeeded": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_write_failed": 0 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log": { 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evlrg": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evadd": 199338, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evex": 108422, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evtrm": 98321, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "ev": 101017, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evexg": 0, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "evexd": 10101, 2024-09-21T03:11:41.774 INFO:teuthology.orchestra.run.smithi018.stdout: "segadd": 284, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "segex": 156, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "segtrm": 141, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "seg": 143, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "segmjr": 9, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "segexg": 0, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "segexd": 15, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "expos": 286968876, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "wrpos": 824263803, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "rdpos": 0, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "jlat": { 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 6630, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 2017.987896503, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.304372231 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.775 INFO:teuthology.orchestra.run.smithi018.stdout: "replayed": 0 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mem": { 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 120, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "ino+": 157, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "ino-": 37, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dir": 22, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dir+": 22, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dir-": 0, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dn": 146, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dn+": 164, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "dn-": 18, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "cap": 4, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "cap+": 37, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "cap-": 33, 2024-09-21T03:11:41.776 INFO:teuthology.orchestra.run.smithi018.stdout: "rss": 145348, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "heap": 403582976 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_server": { 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_client_request": 481, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_server_request": 0, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_request": 271, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_session": 25, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_peer_request": 0, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "req_create_latency": { 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 22, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.022640150, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001029097 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getattr_latency": { 2024-09-21T03:11:41.777 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 170, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 23.819002720, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.140111780 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getfilelock_latency": { 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "req_link_latency": { 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.778 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookup_latency": { 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookuphash_latency": { 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupino_latency": { 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2, 2024-09-21T03:11:41.779 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000428542, 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000214271 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupname_latency": { 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.780 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupparent_latency": { 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupsnap_latency": { 2024-09-21T03:11:41.781 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lssnap_latency": { 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 11, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.001906189, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000173289 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mkdir_latency": { 2024-09-21T03:11:41.782 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 10, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.013658722, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001365872 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mknod_latency": { 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mksnap_latency": { 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 11, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 5.447179495, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.495198135 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "req_open_latency": { 2024-09-21T03:11:41.783 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000245832, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000245832 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "req_readdir_latency": { 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 5, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.016400676, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.003280135 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rename_latency": { 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 2, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.002108834, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001054417 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "req_renamesnap_latency": { 2024-09-21T03:11:41.784 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "req_snapdiff_latency": { 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmdir_latency": { 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 4, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.622559803, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.155639950 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.785 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmsnap_latency": { 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 5, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.718783725, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.143756745 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmxattr_latency": { 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setattr_latency": { 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.001490522, 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001490522 2024-09-21T03:11:41.786 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setdirlayout_latency": { 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setfilelock_latency": { 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setlayout_latency": { 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.787 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setxattr_latency": { 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 9, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.022325916, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.002480657 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "req_symlink_latency": { 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "req_unlink_latency": { 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 18, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.024299890, 2024-09-21T03:11:41.788 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.001349993 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getvxattr_latency": { 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessions": { 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "session_count": 2, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "session_add": 3, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "session_remove": 1, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_open": 2, 2024-09-21T03:11:41.789 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_stale": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "total_load": 49, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "average_load": 24, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "avg_session_uptime": 665, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "mdthresh_evicted": 0 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:41.790 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:41.791 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 336725, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 138, 2024-09-21T03:11:41.792 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 176, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 2, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 8264, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 183, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 463216, 2024-09-21T03:11:41.793 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 3581, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter": { 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_active": 0, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_laggy": 0, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send": 11067, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send_bytes": 820249569, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_resend": 37, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_reply": 11067, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "op_latency": { 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 11067, 2024-09-21T03:11:41.794 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 2199.786640573, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.198769914 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op_inflight": 0, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "oplen_avg": { 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 11067, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 11199 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op": 11067, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op_r": 602, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op_w": 10465, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op_rmw": 0, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "op_pg": 0, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_stat": 1, 2024-09-21T03:11:41.795 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_create": 44, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_read": 20, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_write": 6631, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writefull": 56, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writesame": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_append": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_zero": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_truncate": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_delete": 3352, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_mapext": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_sparse_read": 0, 2024-09-21T03:11:41.796 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_clonerange": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_getxattr": 582, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_setxattr": 88, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_rmxattr": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_call": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_watch": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_notify": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_other": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_active": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_send": 0, 2024-09-21T03:11:41.797 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_resend": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_ping": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_active": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_send": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_resend": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_active": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_send": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_resend": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_active": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_send": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_resend": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "command_active": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "command_send": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "command_resend": 0, 2024-09-21T03:11:41.798 INFO:teuthology.orchestra.run.smithi018.stdout: "map_epoch": 89, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "map_full": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "map_inc": 14, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_sessions": 12, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_open": 12, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_close": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_laggy": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_wr": 408, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_rd": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_del": 17 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "oft": { 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_objs": 1, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_updates": 0, 2024-09-21T03:11:41.799 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_removes": 0 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "purge_queue": { 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops": 0, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops_high_water": 518, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing": 0, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_high_water": 6, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed_ops": 2578, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed": 18, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_item_in_journal": 0 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 211857, 2024-09-21T03:11:41.800 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 54870768, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 211857, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 211857, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 54870768, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_bytes": { 2024-09-21T03:11:41.801 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:41.802 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 11067, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 820252251, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 11067, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 820252251, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_ops": { 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:41.803 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 1024, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 11067, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 11067, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 11067, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 11067, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.804 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 18, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 2682, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 18, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 1, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 2682, 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:41.805 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle-0x55cd0e1ed450": { 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 199338, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 820069499, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 199338, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:41.806 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 6630, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 820069499, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "trackedop": { 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_ops_count": 1 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:41.807 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-21T03:11:42.827 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-09-21T03:11:42.827 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=11 2024-09-21T03:11:42.827 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=5 2024-09-21T03:11:42.827 DEBUG:tasks.check_counter:Getting stats from e 2024-09-21T03:11:42.827 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.e perf dump 2024-09-21T03:11:42.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:42 smithi018 ceph-mon[34587]: pgmap v671: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 140 op/s 2024-09-21T03:11:42.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:42 smithi123 ceph-mon[34645]: pgmap v671: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 140 op/s 2024-09-21T03:11:43.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:42 smithi016 ceph-mon[31253]: pgmap v671: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 7.0 KiB/s rd, 6.4 KiB/s wr, 140 op/s 2024-09-21T03:11:43.090 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:43.636 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.001284161, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000431695, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:43.637 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 231, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 224, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 842384, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 136832, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.042548251, 2024-09-21T03:11:43.638 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.015518603, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.036760010, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 224, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.017632397, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000078716 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 842384, 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 136832 2024-09-21T03:11:43.639 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 168, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 109760, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.012480459, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.009807297, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.001118560, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 166, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.008061594, 2024-09-21T03:11:43.640 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000048563 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 109760 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-09-21T03:11:43.641 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:43.642 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:43.643 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 150015, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 31, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:43.644 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:43.645 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 228, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 807346, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 228, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 228, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 807346, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:43.646 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-21T03:11:44.198 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.e 2024-09-21T03:11:44.199 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.e 2024-09-21T03:11:44.199 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.e 2024-09-21T03:11:44.199 DEBUG:tasks.check_counter:Getting stats from h 2024-09-21T03:11:44.199 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.h perf dump 2024-09-21T03:11:44.464 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:44.868 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:44 smithi018 ceph-mon[34587]: pgmap v672: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 120 op/s 2024-09-21T03:11:44.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:44 smithi123 ceph-mon[34645]: pgmap v672: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 120 op/s 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1873, 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 1866, 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 248885496, 2024-09-21T03:11:44.990 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 655790, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 7, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 3, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.373692070, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.096751629, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.223216154, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.048658578, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1865, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.076082431, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000040794 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:44.991 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 860128, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 138400 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 1297, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 1463, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 342370382, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 791794, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 7, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 6, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.308437666, 2024-09-21T03:11:44.992 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.056830145, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.204596364, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.036430675, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 1461, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.049050167, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000033573 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 379600 2024-09-21T03:11:44.993 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 825, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 825, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 230965693, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 261533, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 6, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 5, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.230945676, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.031075363, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.165571246, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.026015872, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 824, 2024-09-21T03:11:44.994 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.036533232, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000044336 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 1, 2024-09-21T03:11:44.995 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-MDSRank": { 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 5470, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.147963264, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000027049 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "finisher-PurgeQueue": { 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "queue_len": 0, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "complete_latency": { 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:44.996 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "mds": { 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "request": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "reply": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "reply_latency": { 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_reply": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "forward": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_complete": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_fetch_keys": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_commit": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_split": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_merge": 0, 2024-09-21T03:11:44.997 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes": 154, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_top": 24, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_bottom": 130, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pin_tail": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_pinned": 120, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_expired": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "inodes_with_caps": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "caps": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "subtrees": 2, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_hit": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_forward": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_discover": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_dir_fetch": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_remote_ino": 0, 2024-09-21T03:11:44.998 INFO:teuthology.orchestra.run.smithi018.stdout: "traverse_lock": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "load_cent": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "q": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "exported": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "exported_inodes": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "imported": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "imported_inodes": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_dir_fetch": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_backtrace_fetch": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "openino_peer_discover": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rfiles": 4, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rbytes": 234, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "root_rsnaps": 6, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_fetch": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_set_tag": 0, 2024-09-21T03:11:44.999 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_backtrace_repaired": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_inotable_repaired": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_inodes": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dir_base_inodes": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_dirfrag_rstats": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "scrub_file_inodes": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_inode_file_caps": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_revoke": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_grant": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_trunc": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "ceph_cap_op_flush_ack": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_caps_dirty": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_cap_release": 0, 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: "process_request_cap_release": 0 2024-09-21T03:11:45.000 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_cache": { 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_update_receipt": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_try_discover": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_send_discover": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "dir_handle_discover": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_delayed": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_strays_enqueuing": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_created": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_enqueued": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_reintegrated": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "strays_migrated": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_processing": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_enqueued": 0, 2024-09-21T03:11:45.001 INFO:teuthology.orchestra.run.smithi018.stdout: "num_recovering_prioritized": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_started": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "recovery_completed": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_path": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_quiesce_inode": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_enqueue_scrub": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_exportdir": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_flush": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragmentdir": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_fragstats": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "ireq_inodestats": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_started": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_succeeded": 0, 2024-09-21T03:11:45.002 INFO:teuthology.orchestra.run.smithi018.stdout: "uninline_write_failed": 0 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_log": { 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evlrg": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evadd": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evex": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evtrm": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "ev": 199339, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evexg": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "evexd": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "segadd": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "segex": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "segtrm": 0, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "seg": 143, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "segmjr": 18, 2024-09-21T03:11:45.003 INFO:teuthology.orchestra.run.smithi018.stdout: "segexg": 0, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "segexd": 0, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "expos": 286968876, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "wrpos": 824266408, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "rdpos": 824263803, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "jlat": { 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "replayed": 199339 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_mem": { 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "ino": 111, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "ino+": 129, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "ino-": 18, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "dir": 13, 2024-09-21T03:11:45.004 INFO:teuthology.orchestra.run.smithi018.stdout: "dir+": 13, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dir-": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dn": 154, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dn+": 154, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dn-": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "cap": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "cap+": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "cap-": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "rss": 122200, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "heap": 335831040 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_server": { 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_client_request": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "dispatch_server_request": 0, 2024-09-21T03:11:45.005 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_request": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_client_session": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "handle_peer_request": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "req_create_latency": { 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getattr_latency": { 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getfilelock_latency": { 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.006 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "req_link_latency": { 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookup_latency": { 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.007 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookuphash_latency": { 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupino_latency": { 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupname_latency": { 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.008 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupparent_latency": { 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lookupsnap_latency": { 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.009 INFO:teuthology.orchestra.run.smithi018.stdout: "req_lssnap_latency": { 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mkdir_latency": { 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.010 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mknod_latency": { 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "req_mksnap_latency": { 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "req_open_latency": { 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.011 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "req_readdir_latency": { 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rename_latency": { 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "req_renamesnap_latency": { 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.012 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "req_snapdiff_latency": { 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmdir_latency": { 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmsnap_latency": { 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.013 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "req_rmxattr_latency": { 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setattr_latency": { 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setdirlayout_latency": { 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.014 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setfilelock_latency": { 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setlayout_latency": { 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.015 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "req_setxattr_latency": { 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "req_symlink_latency": { 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.016 INFO:teuthology.orchestra.run.smithi018.stdout: "req_unlink_latency": { 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_revoke_eviction": 0, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "cap_acquisition_throttle": 0, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "req_getvxattr_latency": { 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_sessions": { 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "session_count": 2, 2024-09-21T03:11:45.017 INFO:teuthology.orchestra.run.smithi018.stdout: "session_add": 3, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "session_remove": 1, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_open": 2, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "sessions_stale": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "total_load": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "average_load": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "avg_session_uptime": 2026, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mdthresh_evicted": 0 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:45.018 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:45.019 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:45.020 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 242902, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 191, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 8264, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 183, 2024-09-21T03:11:45.021 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 445581, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 4124, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "objecter": { 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "op_active": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "op_laggy": 0, 2024-09-21T03:11:45.022 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send": 3744, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "op_send_bytes": 26, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "op_resend": 4, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "op_reply": 3744, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "op_latency": { 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 3744, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 92.296952774, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.024651963 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "op_inflight": 0, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "oplen_avg": { 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 3744, 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 3745 2024-09-21T03:11:45.023 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "op": 3744, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "op_r": 3744, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "op_w": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "op_rmw": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "op_pg": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_stat": 2501, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_create": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_read": 1240, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_write": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writefull": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_writesame": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_append": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_zero": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_truncate": 0, 2024-09-21T03:11:45.024 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_delete": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_mapext": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_sparse_read": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_clonerange": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_getxattr": 2, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_setxattr": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_cmpxattr": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_rmxattr": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_resetxattrs": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_call": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_watch": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_notify": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_src_cmpxattr": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_pgls_filter": 0, 2024-09-21T03:11:45.025 INFO:teuthology.orchestra.run.smithi018.stdout: "osdop_other": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_active": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_send": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_resend": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "linger_ping": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_active": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_send": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolop_resend": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_active": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_send": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "poolstat_resend": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_active": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_send": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "statfs_resend": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "command_active": 0, 2024-09-21T03:11:45.026 INFO:teuthology.orchestra.run.smithi018.stdout: "command_send": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "command_resend": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "map_epoch": 89, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "map_full": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "map_inc": 14, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_sessions": 11, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_open": 11, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_session_close": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_laggy": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_wr": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_rd": 2, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_del": 0 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "oft": { 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_objs": 0, 2024-09-21T03:11:45.027 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_kv_pairs": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_updates": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "omap_total_removes": 0 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "purge_queue": { 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_ops_high_water": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executing_high_water": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed_ops": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_executed": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "pq_item_in_journal": 0 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:45.028 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 3991, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 821846902, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 3991, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 3991, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 821846902, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.029 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_bytes": { 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 3744, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 820340278, 2024-09-21T03:11:45.030 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 3744, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 820340278, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-objecter_ops": { 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 1024, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:45.031 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 3744, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 3744, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 3744, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 3744, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle": { 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:45.032 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-write_buf_throttle-0x55849ea9d450": { 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:45.033 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 3758096384, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.034 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: "trackedop": { 2024-09-21T03:11:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: "slow_ops_count": 0 2024-09-21T03:11:45.035 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:45.035 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-21T03:11:45.035 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:44 smithi016 ceph-mon[31253]: pgmap v672: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.0 KiB/s rd, 5.5 KiB/s wr, 120 op/s 2024-09-21T03:11:45.824 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-09-21T03:11:45.824 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-09-21T03:11:45.824 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-09-21T03:11:45.824 DEBUG:tasks.check_counter:Getting stats from k 2024-09-21T03:11:45.824 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.k perf dump 2024-09-21T03:11:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:45 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:45.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:45 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config dump", "format": "json"} : dispatch 2024-09-21T03:11:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:46.015 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:46.016 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:45 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:46.081 INFO:teuthology.orchestra.run.smithi018.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.b/config 2024-09-21T03:11:46.574 INFO:teuthology.orchestra.run.smithi018.stdout:{ 2024-09-21T03:11:46.574 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:46.574 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 230, 2024-09-21T03:11:46.574 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 225, 2024-09-21T03:11:46.574 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 840256, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 138304, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.042637924, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.019400800, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.030920036, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 224, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.011326238, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000050563 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:46.575 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 840256, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 138304 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 5, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 173, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 3104, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 110736, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.015172731, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.010555387, 2024-09-21T03:11:46.576 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.002982661, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 172, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.011689877, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000067964 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 110736 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.577 INFO:teuthology.orchestra.run.smithi018.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_total_time": 0.000404012, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_recv_time": 0.000326495, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:46.578 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "cct": { 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "total_workers": 0, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "unhealthy_workers": 0 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "mempool": { 2024-09-21T03:11:46.579 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:46.580 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_items": 0, 2024-09-21T03:11:46.581 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_bytes": 148967, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_anon_items": 29, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:46.582 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_items": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "pgmap_items": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "mds_co_items": 2, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_1_items": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "unittest_2_items": 0 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: }, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "val": 0, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "max": 104857600, 2024-09-21T03:11:46.583 INFO:teuthology.orchestra.run.smithi018.stdout: "get_started": 0, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "get": 231, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "get_sum": 807736, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "get_or_fail_success": 231, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "take": 0, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "take_sum": 0, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "put": 231, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "put_sum": 807736, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "wait": { 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "avgcount": 0, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "sum": 0.000000000, 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: "avgtime": 0.000000000 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout: } 2024-09-21T03:11:46.584 INFO:teuthology.orchestra.run.smithi018.stdout:} 2024-09-21T03:11:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: pgmap v673: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 KiB/s wr, 95 op/s 2024-09-21T03:11:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:47.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:47.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:46 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: pgmap v673: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 KiB/s wr, 95 op/s 2024-09-21T03:11:47.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:47.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:47.221 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:46 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: pgmap v673: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.4 KiB/s rd, 5.6 KiB/s wr, 95 op/s 2024-09-21T03:11:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "config generate-minimal-conf"} : dispatch 2024-09-21T03:11:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "auth get", "entity": "client.admin"} : dispatch 2024-09-21T03:11:47.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:46 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' 2024-09-21T03:11:47.276 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.k 2024-09-21T03:11:47.276 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.k 2024-09-21T03:11:47.276 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.k 2024-09-21T03:11:47.276 DEBUG:tasks.check_counter:Getting stats from c 2024-09-21T03:11:47.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.c perf dump 2024-09-21T03:11:47.545 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:48.084 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001159311, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000278804, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:48.085 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 168, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 109760, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.012631394, 2024-09-21T03:11:48.086 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.009949797, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001029737, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 166, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.007150656, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000043076 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 109760 2024-09-21T03:11:48.087 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 229, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 222, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 841776, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 136032, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.040701307, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.017929741, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.031605038, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 222, 2024-09-21T03:11:48.088 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.014752381, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000066452 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 841776, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 136032 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.089 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:48.090 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:48.091 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148984, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:48.092 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:48.093 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 226, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 807022, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 226, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 226, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 807022, 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:11:48.094 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:48.095 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:48.095 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:11:48.926 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.c 2024-09-21T03:11:48.926 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.c 2024-09-21T03:11:48.926 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.c 2024-09-21T03:11:48.927 DEBUG:tasks.check_counter:Getting stats from f 2024-09-21T03:11:48.927 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.f perf dump 2024-09-21T03:11:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:11:49.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:48 smithi018 ceph-mon[34587]: pgmap v674: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-21T03:11:49.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:48 smithi123 ceph-mon[34645]: pgmap v674: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-21T03:11:49.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:48 smithi016 ceph-mon[31253]: pgmap v674: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 6.2 KiB/s rd, 5.7 KiB/s wr, 6 op/s 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 227, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 222, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 839376, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 136864, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.039890328, 2024-09-21T03:11:49.709 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.018026777, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.029266325, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 221, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011632845, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000052637 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.710 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 839376, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 136864 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 169, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 109936, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.012962665, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.010425684, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000688371, 2024-09-21T03:11:49.711 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 167, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.008514916, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000050987 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 109936 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.712 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.002254793, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000047166, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002205190, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003823267, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001274422 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:49.713 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:49.714 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:49.715 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:49.716 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148984, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 30, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:49.717 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:49.718 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 228, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 807282, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 228, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 228, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 807282, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:49.719 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:49.720 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:49.720 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:49.720 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:11:50.559 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.f 2024-09-21T03:11:50.559 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.f 2024-09-21T03:11:50.559 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.f 2024-09-21T03:11:50.559 DEBUG:tasks.check_counter:Getting stats from i 2024-09-21T03:11:50.559 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.i perf dump 2024-09-21T03:11:50.808 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:11:51.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:50 smithi018 ceph-mon[34587]: pgmap v675: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:51.204 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:50 smithi123 ceph-mon[34645]: pgmap v675: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:51.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:50 smithi016 ceph-mon[31253]: pgmap v675: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:51.325 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 225, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 220, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 838832, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 135584, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.039342699, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.017672680, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.029403554, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:51.326 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 219, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.012461770, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000056903 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 838832, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 135584 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:51.327 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 1, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 168, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 160, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 109760, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.013312723, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.010389156, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.001178157, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 166, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.007872984, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000047427 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:51.328 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 160, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 109760 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 4, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 4, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 2944, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 448, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.002239797, 2024-09-21T03:11:51.329 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000032870, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002190715, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 3, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.003869265, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.001289755 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 2944, 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 448 2024-09-21T03:11:51.330 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:51.331 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:51.332 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 149024, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 31, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:51.333 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:51.334 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 226, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 807022, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 226, 2024-09-21T03:11:51.335 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 226, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 807022, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:51.336 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:11:51.962 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.i 2024-09-21T03:11:51.962 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.i 2024-09-21T03:11:51.962 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.i 2024-09-21T03:11:51.962 DEBUG:tasks.check_counter:Getting stats from l 2024-09-21T03:11:51.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph daemon mds.l perf dump 2024-09-21T03:11:52.221 INFO:teuthology.orchestra.run.smithi123.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.c/config 2024-09-21T03:11:52.766 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-0": { 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 0, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 2, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 0, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 0, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 4, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 0, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.001064910, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.000000000, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.000236362, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:52.767 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 0, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 0 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-1": { 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 232, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 227, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 840800, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 139584, 2024-09-21T03:11:52.768 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 2, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 1, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.040405896, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.019033286, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.030932225, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 226, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011337703, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000050166 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.769 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 840800, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 139584 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "AsyncMessenger::Worker-2": { 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_messages": 5, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages": 175, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_bytes": 3104, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_bytes": 111088, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_created_connections": 3, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_active_connections": 2, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_total_time": 0.014261363, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_send_time": 0.010033643, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_recv_time": 0.002636722, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_messages_queue_lat": { 2024-09-21T03:11:52.770 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 174, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.011389093, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000065454 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_handle_ack_lat": { 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_recv_encrypted_bytes": 3104, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "msgr_send_encrypted_bytes": 111088 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "cct": { 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "total_workers": 0, 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: "unhealthy_workers": 0 2024-09-21T03:11:52.771 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "mempool": { 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bloom_filter_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_alloc_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_data_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_onode_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_meta_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_other_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_bytes": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_cache_buffer_items": 0, 2024-09-21T03:11:52.772 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_extent_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_blob_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_shared_blob_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_inline_bl_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_fsck_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_txc_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_deferred_items": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_bytes": 0, 2024-09-21T03:11:52.773 INFO:teuthology.orchestra.run.smithi123.stdout: "bluestore_writing_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_reader_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "bluefs_file_writer_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_bytes": 148967, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_anon_items": 29, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "buffer_meta_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_items": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_bytes": 0, 2024-09-21T03:11:52.774 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_mapbl_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_pglog_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_mapping_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "pgmap_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_bytes": 968, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_co_items": 2, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_1_items": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_bytes": 0, 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: "unittest_2_items": 0 2024-09-21T03:11:52.775 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "val": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "max": 104857600, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "get_started": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "get": 233, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "get_sum": 807996, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_fail": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "get_or_fail_success": 233, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "take": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "take_sum": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "put": 233, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "put_sum": 807996, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "wait": { 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "avgcount": 0, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "sum": 0.000000000, 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: "avgtime": 0.000000000 2024-09-21T03:11:52.776 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:52.777 INFO:teuthology.orchestra.run.smithi123.stdout: } 2024-09-21T03:11:52.777 INFO:teuthology.orchestra.run.smithi123.stdout:} 2024-09-21T03:11:53.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:52 smithi018 ceph-mon[34587]: pgmap v676: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:53.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:52 smithi123 ceph-mon[34645]: pgmap v676: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:53.265 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:52 smithi016 ceph-mon[31253]: pgmap v676: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:53.437 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.l 2024-09-21T03:11:53.437 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.l 2024-09-21T03:11:53.437 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.l 2024-09-21T03:11:53.437 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-09-21T03:11:53.454 INFO:tasks.kclient:Unmounting kernel clients... 2024-09-21T03:11:53.454 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:11:53.454 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:11:53.475 DEBUG:teuthology.orchestra.run.smithi016:> set -ex 2024-09-21T03:11:53.475 DEBUG:teuthology.orchestra.run.smithi016:> dd if=/proc/self/mounts of=/dev/stdout 2024-09-21T03:11:53.535 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd blocklist ls 2024-09-21T03:11:53.847 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.331+0000 7f75e03b8640 1 -- 172.21.15.16:0/3787459255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d81094a0 msgr2=0x7f75d81098b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.331+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3787459255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d81094a0 0x7f75d81098b0 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009900 tx=0x7f75cc02a410 comp rx=0 tx=0).stop 2024-09-21T03:11:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 -- 172.21.15.16:0/3787459255 shutdown_connections 2024-09-21T03:11:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3787459255 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d81094a0 0x7f75d81098b0 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.339 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3787459255 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d8109df0 0x7f75d8110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3787459255 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8103b60 0x7f75d8103f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 -- 172.21.15.16:0/3787459255 >> 172.21.15.16:0/3787459255 conn(0x7f75d807b850 msgr2=0x7f75d807bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 -- 172.21.15.16:0/3787459255 shutdown_connections 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 -- 172.21.15.16:0/3787459255 wait complete. 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 Processor -- start 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.332+0000 7f75e03b8640 1 -- start start 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d81094a0 0x7f75d819aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 0x7f75d819fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8113ab0 con 0x7f75d8103b60 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8113930 con 0x7f75d81094a0 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75e03b8640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f75d8113c30 con 0x7f75d8109df0 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51662/0 (socket says 172.21.15.16:51662) 2024-09-21T03:11:54.340 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 -- 172.21.15.16:0/3548532537 learned_addr learned my addr 172.21.15.16:0/3548532537 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75dd953640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d81094a0 0x7f75d819aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de955640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 0x7f75d819fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 -- 172.21.15.16:0/3548532537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 msgr2=0x7f75d819fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 0x7f75d819fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 -- 172.21.15.16:0/3548532537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d81094a0 msgr2=0x7f75d819aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d81094a0 0x7f75d819aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 -- 172.21.15.16:0/3548532537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f75d81a0390 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de955640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 0x7f75d819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75de154640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f75c800d770 tx=0x7f75c800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.333+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8017070 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.334+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f75d81a0680 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.334+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f75c80054d0 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.334+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f75c8004c30 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.334+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f75d81a6ea0 con 0x7f75d8103b60 2024-09-21T03:11:54.341 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.335+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75d8104de0 con 0x7f75d8103b60 2024-09-21T03:11:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.336+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f75c8004dd0 con 0x7f75d8103b60 2024-09-21T03:11:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.337+0000 7f75d77fe640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 0x7f75c00b8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.337+0000 7f75dd953640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 0x7f75c00b8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.337+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f75c80e72c0 con 0x7f75d8103b60 2024-09-21T03:11:54.342 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.338+0000 7f75dd953640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 0x7f75c00b8bc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009230 tx=0x7f75cc031400 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:54.343 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.342+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f75c80ad800 con 0x7f75d8103b60 2024-09-21T03:11:54.546 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.545+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f75d810bc20 con 0x7f75d8103b60 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.545+0000 7f75d77fe640 1 -- 172.21.15.16:0/3548532537 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 15 entries v89) ==== 81+0+885 (secure 0 0 0) 0x7f75c80b1be0 con 0x7f75d8103b60 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/2582521578 2024-09-22T02:51:53.053549+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3267035505 2024-09-22T02:50:23.451106+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3267035505 2024-09-22T02:50:23.451106+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/3734474829 2024-09-22T02:50:50.132929+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3058825331 2024-09-22T02:51:53.053549+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/156976776 2024-09-22T02:50:23.451106+0000 2024-09-21T03:11:54.547 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6800/2582521578 2024-09-22T02:51:53.053549+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/827779984 2024-09-22T02:50:23.451106+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3317207057 2024-09-22T02:50:50.132929+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1099561653 2024-09-22T02:50:50.132929+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/3096378034 2024-09-22T02:50:23.451106+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:6801/3734474829 2024-09-22T02:50:50.132929+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/1180697074 2024-09-22T02:50:50.132929+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/338428959 2024-09-22T02:51:53.053549+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stdout:172.21.15.16:0/2917113064 2024-09-22T02:51:53.053549+0000 2024-09-21T03:11:54.548 INFO:teuthology.orchestra.run.smithi016.stderr:listed 15 entries 2024-09-21T03:11:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 msgr2=0x7f75c00b8bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 0x7f75c00b8bc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75cc009230 tx=0x7f75cc031400 comp rx=0 tx=0).stop 2024-09-21T03:11:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 msgr2=0x7f75d819a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:54.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f75c800d770 tx=0x7f75c800dc30 comp rx=0 tx=0).stop 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 shutdown_connections 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f75c00b66b0 0x7f75c00b8bc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f75d8109df0 0x7f75d819fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f75d81094a0 0x7f75d819aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 --2- 172.21.15.16:0/3548532537 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f75d8103b60 0x7f75d819a4e0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 >> 172.21.15.16:0/3548532537 conn(0x7f75d807b850 msgr2=0x7f75d8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 shutdown_connections 2024-09-21T03:11:54.550 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:54.548+0000 7f75e03b8640 1 -- 172.21.15.16:0/3548532537 wait complete. 2024-09-21T03:11:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:54 smithi018 ceph-mon[34587]: pgmap v677: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:11:55.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:54 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3548532537' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:11:55.189 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:54 smithi016 ceph-mon[31253]: pgmap v677: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:11:55.189 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:54 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3548532537' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:11:55.190 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-09-21T03:11:55.190 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:11:55.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-09-21T03:11:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:54 smithi123 ceph-mon[34645]: pgmap v677: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 341 B/s wr, 1 op/s 2024-09-21T03:11:55.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:54 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3548532537' entity='client.admin' cmd={"prefix": "osd blocklist ls"} : dispatch 2024-09-21T03:11:55.238 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:11:55.238 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:11:55.239 DEBUG:teuthology.orchestra.run.smithi016:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-09-21T03:11:55.282 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:11:55.282 DEBUG:teuthology.orchestra.run.smithi016:> ip netns list 2024-09-21T03:11:55.349 INFO:teuthology.orchestra.run.smithi016.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-09-21T03:11:55.350 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:11:55.350 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-09-21T03:11:55.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-09-21T03:11:55.417 DEBUG:teuthology.orchestra.run.smithi016:> sudo ip link delete ceph-brx 2024-09-21T03:11:55.518 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-09-21T03:11:55.528 INFO:tasks.cephadm:Teardown begin 2024-09-21T03:11:55.528 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-09-21T03:11:55.528 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:11:55.791 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 -- 172.21.15.16:0/4142066186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c102a80 msgr2=0x7fa02c102e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 --2- 172.21.15.16:0/4142066186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c102a80 0x7fa02c102e90 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009900 tx=0x7fa01c02a410 comp rx=0 tx=0).stop 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 -- 172.21.15.16:0/4142066186 shutdown_connections 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 --2- 172.21.15.16:0/4142066186 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c108d10 0x7fa02c10f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 --2- 172.21.15.16:0/4142066186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 0x7fa02c1087d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 --2- 172.21.15.16:0/4142066186 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c102a80 0x7fa02c102e90 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 -- 172.21.15.16:0/4142066186 >> 172.21.15.16:0/4142066186 conn(0x7fa02c079740 msgr2=0x7fa02c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:56.285 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 -- 172.21.15.16:0/4142066186 shutdown_connections 2024-09-21T03:11:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 -- 172.21.15.16:0/4142066186 wait complete. 2024-09-21T03:11:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.283+0000 7fa031f5d640 1 Processor -- start 2024-09-21T03:11:56.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 -- start start 2024-09-21T03:11:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c102a80 0x7fa02c19a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 0x7fa02c19aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c113aa0 con 0x7fa02c108d10 2024-09-21T03:11:56.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c113920 con 0x7fa02c102a80 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa031f5d640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fa02c113c20 con 0x7fa02c1083c0 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa02bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa02bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51678/0 (socket says 172.21.15.16:51678) 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa02bfff640 1 -- 172.21.15.16:0/7003248 learned_addr learned my addr 172.21.15.16:0/7003248 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa02b7fe640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c102a80 0x7fa02c19a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.284+0000 7fa02affd640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 0x7fa02c19aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 -- 172.21.15.16:0/7003248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 msgr2=0x7fa02c19aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 0x7fa02c19aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 -- 172.21.15.16:0/7003248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c102a80 msgr2=0x7fa02c19a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c102a80 0x7fa02c19a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 -- 172.21.15.16:0/7003248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa02c1a0440 con 0x7fa02c108d10 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa02bfff640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa02400eaa0 tx=0x7fa02400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa02400cdb0 con 0x7fa02c108d10 2024-09-21T03:11:56.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa024005020 con 0x7fa02c108d10 2024-09-21T03:11:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa02c1a0730 con 0x7fa02c108d10 2024-09-21T03:11:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0240116d0 con 0x7fa02c108d10 2024-09-21T03:11:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.285+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa02c1a6e90 con 0x7fa02c108d10 2024-09-21T03:11:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.286+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa02c103d00 con 0x7fa02c108d10 2024-09-21T03:11:56.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.288+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fa0240051d0 con 0x7fa02c108d10 2024-09-21T03:11:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.289+0000 7fa028ff9640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 0x7fa0140b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.289+0000 7fa02b7fe640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 0x7fa0140b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:56.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.289+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7fa0240e7c50 con 0x7fa02c108d10 2024-09-21T03:11:56.291 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.289+0000 7fa02b7fe640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 0x7fa0140b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa01c031440 tx=0x7fa01c009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:56.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.292+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fa024016030 con 0x7fa02c108d10 2024-09-21T03:11:56.521 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.520+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa02c102e90 con 0x7fa02c108d10 2024-09-21T03:11:56.522 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.521+0000 7fa028ff9640 1 -- 172.21.15.16:0/7003248 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7fa02c102e90 con 0x7fa02c108d10 2024-09-21T03:11:56.523 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:56.523 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:11:56.523 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:11:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 msgr2=0x7fa0140b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:56.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 0x7fa0140b8b90 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fa01c031440 tx=0x7fa01c009560 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 msgr2=0x7fa02c19fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fa02400eaa0 tx=0x7fa02400ef60 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 shutdown_connections 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fa0140b6680 0x7fa0140b8b90 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fa02c108d10 0x7fa02c19fd10 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa02c1083c0 0x7fa02c19aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 --2- 172.21.15.16:0/7003248 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fa02c102a80 0x7fa02c19a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 >> 172.21.15.16:0/7003248 conn(0x7fa02c079740 msgr2=0x7fa02c101500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 shutdown_connections 2024-09-21T03:11:56.526 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:56.524+0000 7fa031f5d640 1 -- 172.21.15.16:0/7003248 wait complete. 2024-09-21T03:11:57.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:56 smithi016 ceph-mon[31253]: pgmap v678: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:57.137 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:56 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/7003248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:57.171 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs dump --format=json 2024-09-21T03:11:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:56 smithi018 ceph-mon[34587]: pgmap v678: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:57.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:56 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/7003248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:56 smithi123 ceph-mon[34645]: pgmap v678: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.5 KiB/s rd, 341 B/s wr, 2 op/s 2024-09-21T03:11:57.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:56 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/7003248' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:57.456 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:57.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 -- 172.21.15.16:0/439048647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 msgr2=0x7f4c6410f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:57.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 --2- 172.21.15.16:0/439048647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c6410f550 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c009ca0 tx=0x7f4c5c02a8c0 comp rx=0 tx=0).stop 2024-09-21T03:11:57.996 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 -- 172.21.15.16:0/439048647 shutdown_connections 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 --2- 172.21.15.16:0/439048647 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c6410f550 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 --2- 172.21.15.16:0/439048647 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 0x7f4c641087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 --2- 172.21.15.16:0/439048647 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c64102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 -- 172.21.15.16:0/439048647 >> 172.21.15.16:0/439048647 conn(0x7f4c640797b0 msgr2=0x7f4c64079bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.994+0000 7f4c6a270640 1 -- 172.21.15.16:0/439048647 shutdown_connections 2024-09-21T03:11:57.997 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.995+0000 7f4c6a270640 1 -- 172.21.15.16:0/439048647 wait complete. 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.995+0000 7f4c6a270640 1 Processor -- start 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.995+0000 7f4c6a270640 1 -- start start 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c6419c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 0x7f4c6419cca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c641a1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:57.999 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f4c641129e0 con 0x7f4c64108d20 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64112860 con 0x7f4c641083d0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6a270640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f4c64112b60 con 0x7f4c64102ae0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6880d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c641a1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c6419c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6880d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c641a1f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51702/0 (socket says 172.21.15.16:51702) 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c6880d640 1 -- 172.21.15.16:0/629211335 learned_addr learned my addr 172.21.15.16:0/629211335 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c637fe640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 0x7f4c6419cca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 -- 172.21.15.16:0/629211335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 msgr2=0x7f4c6419cca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 0x7f4c6419cca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 -- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 msgr2=0x7f4c641a1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c641a1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.996+0000 7f4c63fff640 1 -- 172.21.15.16:0/629211335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c641a25b0 con 0x7f4c64102ae0 2024-09-21T03:11:58.000 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c63fff640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c6419c700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400d720 tx=0x7f4c5400db20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54017070 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c540054d0 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c641a28a0 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c54009e50 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.997+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c641a90c0 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:57.998+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c64102ef0 con 0x7f4c64102ae0 2024-09-21T03:11:58.001 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.000+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f4c54005020 con 0x7f4c64102ae0 2024-09-21T03:11:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.000+0000 7f4c617fa640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 0x7f4c4c0b8de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.001+0000 7f4c637fe640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 0x7f4c4c0b8de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:58.002 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.001+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f4c540e74d0 con 0x7f4c64102ae0 2024-09-21T03:11:58.003 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.001+0000 7f4c637fe640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 0x7f4c4c0b8de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4c64108cb0 tx=0x7f4c5800a2d0 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:58.005 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.004+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f4c54020020 con 0x7f4c64102ae0 2024-09-21T03:11:58.232 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.231+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4c64100c90 con 0x7f4c64102ae0 2024-09-21T03:11:58.233 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.232+0000 7f4c617fa640 1 -- 172.21.15.16:0/629211335 <== mon.1 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 20 v20) ==== 94+0+11850 (secure 0 0 0) 0x7f4c540ada90 con 0x7f4c64102ae0 2024-09-21T03:11:58.233 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:58.234 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":20,"btime":"2024-09-21T02:58:37:670718+0000","default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14772,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6835/3714505775","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6834","nonce":3714505775},{"type":"v1","addr":"172.21.15.16:6835","nonce":3714505775}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14784,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6837/2928144675","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6836","nonce":2928144675},{"type":"v1","addr":"172.21.15.16:6837","nonce":2928144675}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14796,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6835/3014746634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6834","nonce":3014746634},{"type":"v1","addr":"172.21.15.18:6835","nonce":3014746634}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":14844,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6839/2173086680","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6838","nonce":2173086680},{"type":"v1","addr":"172.21.15.16:6839","nonce":2173086680}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14850,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.16:6841/3872585849","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6840","nonce":3872585849},{"type":"v1","addr":"172.21.15.16:6841","nonce":3872585849}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24604,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6833/3810750401","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":3810750401},{"type":"v1","addr":"172.21.15.123:6833","nonce":3810750401}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24617,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.18:6833/3267677843","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6832","nonce":3267677843},{"type":"v1","addr":"172.21.15.18:6833","nonce":3267677843}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24643,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/4189028695","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":4189028695},{"type":"v1","addr":"172.21.15.123:6835","nonce":4189028695}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24655,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2688429163","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2688429163},{"type":"v1","addr":"172.21.15.123:6837","nonce":2688429163}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24664,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6839/1332368204","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6838","nonce":1332368204},{"type":"v1","addr":"172.21.15.123:6839","nonce":1332368204}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":20,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-09-21T02:58:28.365002+0000","modified":"2024-09-21T02:58:37.482868+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24701},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24674":{"gid":24674,"name":"h","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.18:6837/2180643175","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6836","nonce":2180643175},{"type":"v1","addr":"172.21.15.18:6837","nonce":2180643175}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24701":{"gid":24701,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":4,"addr":"172.21.15.18:6839/253530783","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6838","nonce":253530783},{"type":"v1","addr":"172.21.15.18:6839","nonce":253530783}]},"join_fscid":-1,"export_targets":[],"features":4540719139924082687,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24701,"qdb_cluster":[24701]},"id":1}]} 2024-09-21T03:11:58.234 INFO:teuthology.orchestra.run.smithi016.stderr:dumped fsmap epoch 20 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 msgr2=0x7f4c4c0b8de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 0x7f4c4c0b8de0 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4c64108cb0 tx=0x7f4c5800a2d0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 msgr2=0x7f4c6419c700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c6419c700 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f4c5400d720 tx=0x7f4c5400db20 comp rx=0 tx=0).stop 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 shutdown_connections 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f4c4c0b68d0 0x7f4c4c0b8de0 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f4c64108d20 0x7f4c641a1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f4c641083d0 0x7f4c6419cca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.236 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 --2- 172.21.15.16:0/629211335 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c64102ae0 0x7f4c6419c700 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 >> 172.21.15.16:0/629211335 conn(0x7f4c640797b0 msgr2=0x7f4c6406b9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.234+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 shutdown_connections 2024-09-21T03:11:58.237 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:58.235+0000 7f4c6a270640 1 -- 172.21.15.16:0/629211335 wait complete. 2024-09-21T03:11:58.909 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd dump --format=json 2024-09-21T03:11:59.171 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:11:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:58 smithi018 ceph-mon[34587]: pgmap v679: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:11:59.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:58 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/629211335' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:58 smithi123 ceph-mon[34645]: pgmap v679: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:11:59.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:58 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/629211335' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:59.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:58 smithi016 ceph-mon[31253]: pgmap v679: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 255 B/s wr, 2 op/s 2024-09-21T03:11:59.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:58 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/629211335' entity='client.admin' cmd={"prefix": "fs dump", "format": "json"} : dispatch 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 -- 172.21.15.16:0/1403865191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920102a90 msgr2=0x7f6920102ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 --2- 172.21.15.16:0/1403865191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920102a90 0x7f6920102ea0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f6910009900 tx=0x7f691002a410 comp rx=0 tx=0).stop 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 -- 172.21.15.16:0/1403865191 shutdown_connections 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 --2- 172.21.15.16:0/1403865191 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f6920108d20 0x7f692010f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 --2- 172.21.15.16:0/1403865191 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f69201083d0 0x7f69201087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 --2- 172.21.15.16:0/1403865191 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920102a90 0x7f6920102ea0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.668 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 -- 172.21.15.16:0/1403865191 >> 172.21.15.16:0/1403865191 conn(0x7f6920079750 msgr2=0x7f6920079b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 -- 172.21.15.16:0/1403865191 shutdown_connections 2024-09-21T03:11:59.669 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.666+0000 7f69264f9640 1 -- 172.21.15.16:0/1403865191 wait complete. 2024-09-21T03:11:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 Processor -- start 2024-09-21T03:11:59.670 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 -- start start 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6920102a90 0x7f692019a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69201083d0 0x7f692019aa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f6920112990 con 0x7f6920108d20 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f6920112810 con 0x7f69201083d0 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.667+0000 7f69264f9640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f6920112b10 con 0x7f6920102a90 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51718/0 (socket says 172.21.15.16:51718) 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 -- 172.21.15.16:0/3119727275 learned_addr learned my addr 172.21.15.16:0/3119727275 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f691ffff640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6920102a90 0x7f692019a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6917fff640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69201083d0 0x7f692019aa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 -- 172.21.15.16:0/3119727275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6920102a90 msgr2=0x7f692019a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.671 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6920102a90 0x7f692019a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 -- 172.21.15.16:0/3119727275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69201083d0 msgr2=0x7f692019aa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69201083d0 0x7f692019aa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 -- 172.21.15.16:0/3119727275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69201a0420 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f6924a96640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f691800eaa0 tx=0x7f691800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f691800cdb0 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6918005020 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69201a0710 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.668+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f69180116d0 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.669+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69201a6e70 con 0x7f6920108d20 2024-09-21T03:11:59.672 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.670+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6920103d10 con 0x7f6920108d20 2024-09-21T03:11:59.673 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.672+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f69180051d0 con 0x7f6920108d20 2024-09-21T03:11:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.673+0000 7f691dffb640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 0x7f690c0b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:11:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.673+0000 7f691ffff640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 0x7f690c0b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:11:59.674 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.673+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f69180e7ca0 con 0x7f6920108d20 2024-09-21T03:11:59.675 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.674+0000 7f691ffff640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 0x7f690c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6910031440 tx=0x7f6910009560 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:11:59.677 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.676+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f6918016030 con 0x7f6920108d20 2024-09-21T03:11:59.868 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.867+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6920102ea0 con 0x7f6920108d20 2024-09-21T03:11:59.870 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.868+0000 7f691dffb640 1 -- 172.21.15.16:0/3119727275 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+23217 (secure 0 0 0) 0x7f6920102ea0 con 0x7f6920108d20 2024-09-21T03:11:59.870 INFO:teuthology.orchestra.run.smithi016.stdout: 2024-09-21T03:11:59.870 INFO:teuthology.orchestra.run.smithi016.stdout:{"epoch":89,"fsid":"ffc7910c-77c3-11ef-baf5-efdc52797490","created":"2024-09-21T02:50:00.888391+0000","modified":"2024-09-21T03:11:31.297206+0000","last_up_change":"2024-09-21T02:57:37.079390+0000","last_in_change":"2024-09-21T02:57:14.287812+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-09-21T02:53:55.521347+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-09-21T02:58:24.062466+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"75","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-09-21T02:58:26.393380+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"88","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":17,"snap_epoch":88,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"c38a1c84-7e88-4346-bb53-088922f08bbe","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6802","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6803","nonce":1882705397}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6804","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6805","nonce":1882705397}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6808","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6809","nonce":1882705397}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6806","nonce":1882705397},{"type":"v1","addr":"172.21.15.16:6807","nonce":1882705397}]},"public_addr":"172.21.15.16:6803/1882705397","cluster_addr":"172.21.15.16:6805/1882705397","heartbeat_back_addr":"172.21.15.16:6809/1882705397","heartbeat_front_addr":"172.21.15.16:6807/1882705397","state":["exists","up"]},{"osd":1,"uuid":"ab60e425-6410-41cb-a003-2dc7e21c8cb0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6800","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6801","nonce":1950134733}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6802","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6803","nonce":1950134733}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6806","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6807","nonce":1950134733}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6804","nonce":1950134733},{"type":"v1","addr":"172.21.15.18:6805","nonce":1950134733}]},"public_addr":"172.21.15.18:6801/1950134733","cluster_addr":"172.21.15.18:6803/1950134733","heartbeat_back_addr":"172.21.15.18:6807/1950134733","heartbeat_front_addr":"172.21.15.18:6805/1950134733","state":["exists","up"]},{"osd":2,"uuid":"16bdfeb8-142c-4773-915d-037f7d9f90f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":19,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6801","nonce":3497511537}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6803","nonce":3497511537}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6807","nonce":3497511537}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":3497511537},{"type":"v1","addr":"172.21.15.123:6805","nonce":3497511537}]},"public_addr":"172.21.15.123:6801/3497511537","cluster_addr":"172.21.15.123:6803/3497511537","heartbeat_back_addr":"172.21.15.123:6807/3497511537","heartbeat_front_addr":"172.21.15.123:6805/3497511537","state":["exists","up"]},{"osd":3,"uuid":"8bdffca0-2cd0-41f5-be85-ebe805e4a696","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":26,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6810","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6811","nonce":3760097578}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6812","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6813","nonce":3760097578}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6816","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6817","nonce":3760097578}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6814","nonce":3760097578},{"type":"v1","addr":"172.21.15.16:6815","nonce":3760097578}]},"public_addr":"172.21.15.16:6811/3760097578","cluster_addr":"172.21.15.16:6813/3760097578","heartbeat_back_addr":"172.21.15.16:6817/3760097578","heartbeat_front_addr":"172.21.15.16:6815/3760097578","state":["exists","up"]},{"osd":4,"uuid":"3ee9ba63-3aca-4171-b196-a3b9e633a6b0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":32,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6808","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6809","nonce":2331184101}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6810","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6811","nonce":2331184101}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6814","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6815","nonce":2331184101}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6812","nonce":2331184101},{"type":"v1","addr":"172.21.15.18:6813","nonce":2331184101}]},"public_addr":"172.21.15.18:6809/2331184101","cluster_addr":"172.21.15.18:6811/2331184101","heartbeat_back_addr":"172.21.15.18:6815/2331184101","heartbeat_front_addr":"172.21.15.18:6813/2331184101","state":["exists","up"]},{"osd":5,"uuid":"ff1034fc-1839-47d8-80c6-93e0271bd1ec","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6809","nonce":2715140085}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6811","nonce":2715140085}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6815","nonce":2715140085}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":2715140085},{"type":"v1","addr":"172.21.15.123:6813","nonce":2715140085}]},"public_addr":"172.21.15.123:6809/2715140085","cluster_addr":"172.21.15.123:6811/2715140085","heartbeat_back_addr":"172.21.15.123:6815/2715140085","heartbeat_front_addr":"172.21.15.123:6813/2715140085","state":["exists","up"]},{"osd":6,"uuid":"c0460abd-d09c-42d4-90e1-fb8ce20982f3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":43,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6818","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6819","nonce":3190074295}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6820","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6821","nonce":3190074295}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6824","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6825","nonce":3190074295}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6822","nonce":3190074295},{"type":"v1","addr":"172.21.15.16:6823","nonce":3190074295}]},"public_addr":"172.21.15.16:6819/3190074295","cluster_addr":"172.21.15.16:6821/3190074295","heartbeat_back_addr":"172.21.15.16:6825/3190074295","heartbeat_front_addr":"172.21.15.16:6823/3190074295","state":["exists","up"]},{"osd":7,"uuid":"1197b70e-2896-4524-8174-10c95b6b03c7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":48,"up_thru":73,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6816","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6817","nonce":3027010231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6818","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6819","nonce":3027010231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6822","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6823","nonce":3027010231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6820","nonce":3027010231},{"type":"v1","addr":"172.21.15.18:6821","nonce":3027010231}]},"public_addr":"172.21.15.18:6817/3027010231","cluster_addr":"172.21.15.18:6819/3027010231","heartbeat_back_addr":"172.21.15.18:6823/3027010231","heartbeat_front_addr":"172.21.15.18:6821/3027010231","state":["exists","up"]},{"osd":8,"uuid":"df052aaa-1464-46d8-828f-99ce714b76c6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":54,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6817","nonce":1022177743}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6819","nonce":1022177743}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6823","nonce":1022177743}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":1022177743},{"type":"v1","addr":"172.21.15.123:6821","nonce":1022177743}]},"public_addr":"172.21.15.123:6817/1022177743","cluster_addr":"172.21.15.123:6819/1022177743","heartbeat_back_addr":"172.21.15.123:6823/1022177743","heartbeat_front_addr":"172.21.15.123:6821/1022177743","state":["exists","up"]},{"osd":9,"uuid":"9d9f79be-daa6-4305-8037-4487eb7b3e5a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6826","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6827","nonce":661150471}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6828","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6829","nonce":661150471}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6832","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6833","nonce":661150471}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.16:6830","nonce":661150471},{"type":"v1","addr":"172.21.15.16:6831","nonce":661150471}]},"public_addr":"172.21.15.16:6827/661150471","cluster_addr":"172.21.15.16:6829/661150471","heartbeat_back_addr":"172.21.15.16:6833/661150471","heartbeat_front_addr":"172.21.15.16:6831/661150471","state":["exists","up"]},{"osd":10,"uuid":"84176f3a-eba3-4fa1-aaf6-523a10b9d713","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":64,"up_thru":76,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6824","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6825","nonce":1894161437}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6826","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6827","nonce":1894161437}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6830","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6831","nonce":1894161437}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.18:6828","nonce":1894161437},{"type":"v1","addr":"172.21.15.18:6829","nonce":1894161437}]},"public_addr":"172.21.15.18:6825/1894161437","cluster_addr":"172.21.15.18:6827/1894161437","heartbeat_back_addr":"172.21.15.18:6831/1894161437","heartbeat_front_addr":"172.21.15.18:6829/1894161437","state":["exists","up"]},{"osd":11,"uuid":"340835c3-2eab-478a-9def-47542b5fc7bc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":69,"up_thru":78,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6825","nonce":4172042931}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6827","nonce":4172042931}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6831","nonce":4172042931}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":4172042931},{"type":"v1","addr":"172.21.15.123:6829","nonce":4172042931}]},"public_addr":"172.21.15.123:6825/4172042931","cluster_addr":"172.21.15.123:6827/4172042931","heartbeat_back_addr":"172.21.15.123:6831/4172042931","heartbeat_front_addr":"172.21.15.123:6829/4172042931","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:01.010291+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:26.340141+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:53:51.677302+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:17.640817+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:54:42.900596+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:07.072705+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:32.963464+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:55:57.947681+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:21.979009+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:56:47.096072+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:11.146823+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540719139924082687,"old_weight":0,"last_purged_snaps_scrub":"2024-09-21T02:57:36.032702+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.b","mappings":[{"from":2,"to":1}]},{"pgid":"2.18","mappings":[{"from":5,"to":11}]},{"pgid":"2.1a","mappings":[{"from":8,"to":4}]},{"pgid":"2.1d","mappings":[{"from":2,"to":4},{"from":8,"to":11}]},{"pgid":"3.0","mappings":[{"from":5,"to":9}]},{"pgid":"3.6","mappings":[{"from":4,"to":6}]},{"pgid":"3.17","mappings":[{"from":4,"to":8}]},{"pgid":"3.1e","mappings":[{"from":3,"to":9}]},{"pgid":"3.24","mappings":[{"from":4,"to":9}]},{"pgid":"3.28","mappings":[{"from":3,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.16:6801/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:6800/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3267035505":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3058825331":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/156976776":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6800/2582521578":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/827779984":"2024-09-22T02:50:23.451106+0000","172.21.15.16:0/3317207057":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1099561653":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/3096378034":"2024-09-22T02:50:23.451106+0000","172.21.15.16:6801/3734474829":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/1180697074":"2024-09-22T02:50:50.132929+0000","172.21.15.16:0/338428959":"2024-09-22T02:51:53.053549+0000","172.21.15.16:0/2917113064":"2024-09-22T02:51:53.053549+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":6,"length":1},{"begin":17,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-09-21T03:11:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 msgr2=0x7f690c0b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 0x7f690c0b8b90 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f6910031440 tx=0x7f6910009560 comp rx=0 tx=0).stop 2024-09-21T03:11:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 msgr2=0x7f692019fcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:11:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f691800eaa0 tx=0x7f691800ef60 comp rx=0 tx=0).stop 2024-09-21T03:11:59.872 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 shutdown_connections 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f690c0b6680 0x7f690c0b8b90 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f6920108d20 0x7f692019fcf0 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f69201083d0 0x7f692019aa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 --2- 172.21.15.16:0/3119727275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6920102a90 0x7f692019a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.870+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 >> 172.21.15.16:0/3119727275 conn(0x7f6920079750 msgr2=0x7f6920101a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.871+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 shutdown_connections 2024-09-21T03:11:59.873 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:11:59.871+0000 7f69264f9640 1 -- 172.21.15.16:0/3119727275 wait complete. 2024-09-21T03:12:00.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:11:59 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3119727275' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:12:00.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:11:59 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3119727275' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:12:00.227 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:11:59 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3119727275' entity='client.admin' cmd={"prefix": "osd dump", "format": "json"} : dispatch 2024-09-21T03:12:00.544 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs fail cephfs 2024-09-21T03:12:00.824 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:01.190 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:00 smithi016 ceph-mon[31253]: pgmap v680: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T03:12:01.220 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:00 smithi123 ceph-mon[34645]: pgmap v680: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/1423358914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 msgr2=0x7f7a34112620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/1423358914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a34112620 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c009ca0 tx=0x7f7a2c02a950 comp rx=0 tx=0).stop 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/1423358914 shutdown_connections 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/1423358914 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a34112620 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/1423358914 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102fa0 0x7f7a3410fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/1423358914 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102650 0x7f7a34102a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/1423358914 >> 172.21.15.16:0/1423358914 conn(0x7f7a34079760 msgr2=0x7f7a34079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/1423358914 shutdown_connections 2024-09-21T03:12:01.284 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.282+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/1423358914 wait complete. 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 Processor -- start 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 -- start start 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102650 0x7f7a3419c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 0x7f7a3419cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a341a1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34115ca0 con 0x7f7a34110200 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34115b20 con 0x7f7a34102650 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a3b7fc640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7f7a34115e20 con 0x7f7a34102fa0 2024-09-21T03:12:01.286 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a39d99640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a341a1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a38d97640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 0x7f7a3419cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a38d97640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 0x7f7a3419cbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.16:39632/0 (socket says 172.21.15.16:39632) 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a38d97640 1 -- 172.21.15.16:0/2172597815 learned_addr learned my addr 172.21.15.16:0/2172597815 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.283+0000 7f7a39598640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102650 0x7f7a3419c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 -- 172.21.15.16:0/2172597815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 msgr2=0x7f7a3419cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 0x7f7a3419cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 -- 172.21.15.16:0/2172597815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102650 msgr2=0x7f7a3419c610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102650 0x7f7a3419c610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 -- 172.21.15.16:0/2172597815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7a341a2580 con 0x7f7a34110200 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a39d99640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a341a1e50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c005cc0 tx=0x7f7a2c02f650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c034040 con 0x7f7a34110200 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7a2c02fe80 con 0x7f7a34110200 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7a341a2810 con 0x7f7a34110200 2024-09-21T03:12:01.287 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7a2c004670 con 0x7f7a34110200 2024-09-21T03:12:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.284+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7a341a8fd0 con 0x7f7a34110200 2024-09-21T03:12:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.285+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7a3406c1f0 con 0x7f7a34110200 2024-09-21T03:12:01.288 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.287+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7f7a2c037020 con 0x7f7a34110200 2024-09-21T03:12:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.288+0000 7f7a2a7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 0x7f7a200b8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.288+0000 7f7a39598640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 0x7f7a200b8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:01.289 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.288+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8536+0+0 (secure 0 0 0) 0x7f7a2c1085e0 con 0x7f7a34110200 2024-09-21T03:12:01.290 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.288+0000 7f7a39598640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 0x7f7a200b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005fd0 tx=0x7f7a3000b390 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:01.293 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.291+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7f7a2c083710 con 0x7f7a34110200 2024-09-21T03:12:01.432 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:00 smithi018 ceph-mon[34587]: pgmap v680: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.2 KiB/s rd, 1 op/s 2024-09-21T03:12:01.523 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.521+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7f7a341a91a0 con 0x7f7a34110200 2024-09-21T03:12:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.524+0000 7f7a2a7fc640 1 -- 172.21.15.16:0/2172597815 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v21) ==== 161+0+0 (secure 0 0 0) 0x7f7a2c0ceb20 con 0x7f7a34110200 2024-09-21T03:12:01.525 INFO:teuthology.orchestra.run.smithi016.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-09-21T03:12:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.526+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 msgr2=0x7f7a200b8c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.526+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 0x7f7a200b8c10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f7a30005fd0 tx=0x7f7a3000b390 comp rx=0 tx=0).stop 2024-09-21T03:12:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.526+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 msgr2=0x7f7a341a1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:01.528 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.526+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a341a1e50 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f7a2c005cc0 tx=0x7f7a2c02f650 comp rx=0 tx=0).stop 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 shutdown_connections 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7f7a200b6700 0x7f7a200b8c10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7f7a34110200 0x7f7a341a1e50 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a34102fa0 0x7f7a3419cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 --2- 172.21.15.16:0/2172597815 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7f7a34102650 0x7f7a3419c610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 >> 172.21.15.16:0/2172597815 conn(0x7f7a34079760 msgr2=0x7f7a34109d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 shutdown_connections 2024-09-21T03:12:01.529 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:01.527+0000 7f7a3b7fc640 1 -- 172.21.15.16:0/2172597815 wait complete. 2024-09-21T03:12:01.886 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a[31249]: 2024-09-21T03:12:01.522+0000 7f7778d61640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:12:01.932 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: ignoring --setuser ceph since I am not root 2024-09-21T03:12:01.932 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: ignoring --setgroup ceph since I am not root 2024-09-21T03:12:01.932 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-mds[62870]: -- 172.21.15.18:0/1184084649 <== mon.0 v2:172.21.15.16:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x559c41efdc20 con 0x559c460a5a80 2024-09-21T03:12:01.933 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: starting mds.b at 2024-09-21T03:12:01.933 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: ignoring --setuser ceph since I am not root 2024-09-21T03:12:01.933 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: ignoring --setgroup ceph since I am not root 2024-09-21T03:12:01.933 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-mds[62603]: -- 172.21.15.18:0/3603540893 <== mon.2 v2:172.21.15.18:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560b22143860 con 0x560b262eba80 2024-09-21T03:12:01.933 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:12:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: starting mds.h at 2024-09-21T03:12:02.225 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph fs rm cephfs --yes-i-really-mean-it 2024-09-21T03:12:02.470 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:12:02.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:02 smithi123 ceph-mon[34645]: pgmap v682: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:02.485 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:02.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:12:02.515 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:12:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:12:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:12:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:12:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:12:02.516 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:02 smithi016 ceph-mon[31253]: pgmap v682: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd={"prefix": "fs fail", "fs_name": "cephfs"} : dispatch 2024-09-21T03:12:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-09-21T03:12:02.682 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-09-21T03:12:02.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2172597815' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-09-21T03:12:02.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: osdmap e90: 12 total, 12 up, 12 in 2024-09-21T03:12:02.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: fsmap cephfs:0/1 10 up:standby, 1 failed 2024-09-21T03:12:02.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:02 smithi018 ceph-mon[34587]: pgmap v682: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.983+0000 7feeab14a640 1 -- 172.21.15.16:0/2494522192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4103b60 msgr2=0x7feea4103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.983+0000 7feeab14a640 1 --2- 172.21.15.16:0/2494522192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4103b60 0x7feea4103f70 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fee98009900 tx=0x7fee9802a410 comp rx=0 tx=0).stop 2024-09-21T03:12:02.985 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 -- 172.21.15.16:0/2494522192 shutdown_connections 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 --2- 172.21.15.16:0/2494522192 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea4109df0 0x7feea4110620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 --2- 172.21.15.16:0/2494522192 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea41094a0 0x7feea41098b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 --2- 172.21.15.16:0/2494522192 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4103b60 0x7feea4103f70 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 -- 172.21.15.16:0/2494522192 >> 172.21.15.16:0/2494522192 conn(0x7feea407b850 msgr2=0x7feea407bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 -- 172.21.15.16:0/2494522192 shutdown_connections 2024-09-21T03:12:02.986 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 -- 172.21.15.16:0/2494522192 wait complete. 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 Processor -- start 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.984+0000 7feeab14a640 1 -- start start 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea4103b60 0x7feea4075710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea41094a0 0x7feea4075c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7feea4113b00 con 0x7feea4109df0 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7feea4113980 con 0x7feea41094a0 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feeab14a640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7feea4113c80 con 0x7feea4103b60 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:51756/0 (socket says 172.21.15.16:51756) 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 -- 172.21.15.16:0/2260800609 learned_addr learned my addr 172.21.15.16:0/2260800609 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea3fff640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea41094a0 0x7feea4075c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:02.988 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea8ee6640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea4103b60 0x7feea4075710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 -- 172.21.15.16:0/2260800609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea4103b60 msgr2=0x7feea4075710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea4103b60 0x7feea4075710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 -- 172.21.15.16:0/2260800609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea41094a0 msgr2=0x7feea4075c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea41094a0 0x7feea4075c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 -- 172.21.15.16:0/2260800609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea41a8770 con 0x7feea4109df0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea96e7640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cad0 tx=0x7fee9c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.985+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00ae20 con 0x7feea4109df0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.986+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee9c005020 con 0x7feea4109df0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.986+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea41a8a60 con 0x7feea4109df0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.986+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee9c00f6b0 con 0x7feea4109df0 2024-09-21T03:12:02.989 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.986+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea41a8f20 con 0x7feea4109df0 2024-09-21T03:12:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.987+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea4103f70 con 0x7feea4109df0 2024-09-21T03:12:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.988+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fee9c0051d0 con 0x7feea4109df0 2024-09-21T03:12:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.989+0000 7feea1ffb640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 0x7fee940b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.989+0000 7feea8ee6640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 0x7fee940b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:02.990 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.989+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8578+0+0 (secure 0 0 0) 0x7fee9c0e7780 con 0x7feea4109df0 2024-09-21T03:12:02.991 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.990+0000 7feea8ee6640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 0x7fee940b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee98005ab0 tx=0x7fee98005a20 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:02.995 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:02.994+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fee9c014030 con 0x7feea4109df0 2024-09-21T03:12:03.222 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.220+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7feea402e640 con 0x7feea4109df0 2024-09-21T03:12:03.537 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.536+0000 7feea1ffb640 1 -- 172.21.15.16:0/2260800609 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v23) ==== 104+0+0 (secure 0 0 0) 0x7fee9c0b2010 con 0x7feea4109df0 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 msgr2=0x7fee940b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 0x7fee940b8b90 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fee98005ab0 tx=0x7fee98005a20 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 msgr2=0x7feea4076190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fee9c00cad0 tx=0x7fee9c00cf90 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 shutdown_connections 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fee940b6680 0x7fee940b8b90 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7feea4109df0 0x7feea4076190 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7feea41094a0 0x7feea4075c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 --2- 172.21.15.16:0/2260800609 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feea4103b60 0x7feea4075710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 >> 172.21.15.16:0/2260800609 conn(0x7feea407b850 msgr2=0x7feea4107530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:03.541 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 shutdown_connections 2024-09-21T03:12:03.542 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:03.539+0000 7feeab14a640 1 -- 172.21.15.16:0/2260800609 wait complete. 2024-09-21T03:12:03.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6836/4267118971,v1:172.21.15.18:6838/4267118971] up:boot 2024-09-21T03:12:03.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: mds.? [v2:172.21.15.18:6837/1528120046,v1:172.21.15.18:6839/1528120046] up:boot 2024-09-21T03:12:03.869 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:12:03.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:12:03.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:12:03.870 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:03 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:12:03.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6836/4267118971,v1:172.21.15.18:6838/4267118971] up:boot 2024-09-21T03:12:03.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: mds.? [v2:172.21.15.18:6837/1528120046,v1:172.21.15.18:6839/1528120046] up:boot 2024-09-21T03:12:03.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:12:03.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:12:03.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:12:03.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:03 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:12:03.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6836/4267118971,v1:172.21.15.18:6838/4267118971] up:boot 2024-09-21T03:12:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: mds.? [v2:172.21.15.18:6837/1528120046,v1:172.21.15.18:6839/1528120046] up:boot 2024-09-21T03:12:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: fsmap cephfs:0/1 12 up:standby, 1 failed 2024-09-21T03:12:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "b"} : dispatch 2024-09-21T03:12:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: from='mgr.14226 172.21.15.16:0/2453451049' entity='mgr.x' cmd={"prefix": "mds metadata", "who": "h"} : dispatch 2024-09-21T03:12:03.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:03 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd={"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} : dispatch 2024-09-21T03:12:04.164 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-09-21T03:12:04.423 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:04.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:04 smithi016 ceph-mon[31253]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:12:04.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:04 smithi016 ceph-mon[31253]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:12:04.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:04 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:12:04.799 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:04 smithi016 ceph-mon[31253]: fsmap 12 up:standby 2024-09-21T03:12:04.800 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:04 smithi016 ceph-mon[31253]: pgmap v683: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:04.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:04 smithi018 ceph-mon[34587]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:12:04.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:04 smithi018 ceph-mon[34587]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:12:04.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:04 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:12:04.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:04 smithi018 ceph-mon[34587]: fsmap 12 up:standby 2024-09-21T03:12:04.933 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:04 smithi018 ceph-mon[34587]: pgmap v683: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.894+0000 7fb972cbb640 1 -- 172.21.15.16:0/2491977468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c108d20 msgr2=0x7fb96c10f550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:04.933 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.894+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2491977468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c108d20 0x7fb96c10f550 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fb964009900 tx=0x7fb96402a410 comp rx=0 tx=0).stop 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 -- 172.21.15.16:0/2491977468 shutdown_connections 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2491977468 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c108d20 0x7fb96c10f550 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2491977468 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c1083d0 0x7fb96c1087e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2491977468 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c102ae0 0x7fb96c102ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 -- 172.21.15.16:0/2491977468 >> 172.21.15.16:0/2491977468 conn(0x7fb96c079760 msgr2=0x7fb96c079b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 -- 172.21.15.16:0/2491977468 shutdown_connections 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 -- 172.21.15.16:0/2491977468 wait complete. 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 Processor -- start 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.895+0000 7fb972cbb640 1 -- start start 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c102ae0 0x7fb96c19a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c108d20 0x7fb96c19fc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:04.934 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c113ab0 con 0x7fb96c1083d0 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c113930 con 0x7fb96c108d20 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fb96c113c30 con 0x7fb96c102ae0 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53014/0 (socket says 172.21.15.16:53014) 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 -- 172.21.15.16:0/2886405154 learned_addr learned my addr 172.21.15.16:0/2886405154 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb971258640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c108d20 0x7fb96c19fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:04.935 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb970a57640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c102ae0 0x7fb96c19a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 -- 172.21.15.16:0/2886405154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c102ae0 msgr2=0x7fb96c19a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c102ae0 0x7fb96c19a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 -- 172.21.15.16:0/2886405154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c108d20 msgr2=0x7fb96c19fc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c108d20 0x7fb96c19fc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 -- 172.21.15.16:0/2886405154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb96c1a0350 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb96bfff640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d770 tx=0x7fb95400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb954017070 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb96c1a0640 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb954009c30 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.896+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb95400f620 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.897+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb96c1a6da0 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.898+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb96c103d10 con 0x7fb96c1083d0 2024-09-21T03:12:04.938 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.899+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fb95401a070 con 0x7fb96c1083d0 2024-09-21T03:12:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.900+0000 7fb969ffb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 0x7fb9580b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.900+0000 7fb970a57640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 0x7fb9580b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.900+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 8578+0+0 (secure 0 0 0) 0x7fb9540e72b0 con 0x7fb96c1083d0 2024-09-21T03:12:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.901+0000 7fb970a57640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 0x7fb9580b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb95c005c40 tx=0x7fb95c00a780 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:04.939 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:04.905+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fb954009de0 con 0x7fb96c1083d0 2024-09-21T03:12:04.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:04 smithi123 ceph-mon[34645]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-09-21T03:12:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:04 smithi123 ceph-mon[34645]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-09-21T03:12:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:04 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2260800609' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-09-21T03:12:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:04 smithi123 ceph-mon[34645]: fsmap 12 up:standby 2024-09-21T03:12:04.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:04 smithi123 ceph-mon[34645]: pgmap v683: 97 pgs: 97 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail; 1.1 KiB/s rd, 204 B/s wr, 2 op/s 2024-09-21T03:12:05.095 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.094+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7fb96c02e640 con 0x7fb96c1083d0 2024-09-21T03:12:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.543+0000 7fb969ffb640 1 -- 172.21.15.16:0/2886405154 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v91) ==== 181+0+0 (secure 0 0 0) 0x7fb9540ad7c0 con 0x7fb96c1083d0 2024-09-21T03:12:05.545 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_metadata' removed 2024-09-21T03:12:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 msgr2=0x7fb9580b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:05.548 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 0x7fb9580b8b90 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb95c005c40 tx=0x7fb95c00a780 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 msgr2=0x7fb96c19a980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d770 tx=0x7fb95400dc30 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 shutdown_connections 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fb9580b6680 0x7fb9580b8b90 secure :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fb95c005c40 tx=0x7fb95c00a780 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fb96c1083d0 0x7fb96c19a980 secure :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb95400d770 tx=0x7fb95400dc30 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fb96c108d20 0x7fb96c19fc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 --2- 172.21.15.16:0/2886405154 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb96c102ae0 0x7fb96c19a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.546+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 >> 172.21.15.16:0/2886405154 conn(0x7fb96c079760 msgr2=0x7fb96c105230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.547+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 shutdown_connections 2024-09-21T03:12:05.549 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:05.547+0000 7fb972cbb640 1 -- 172.21.15.16:0/2886405154 wait complete. 2024-09-21T03:12:05.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:05 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:05.944 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:05 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:05.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:05 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:06.247 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410 shell --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 -- ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-09-21T03:12:06.511 INFO:teuthology.orchestra.run.smithi016.stderr:Inferring config /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/mon.a/config 2024-09-21T03:12:06.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:06 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:06.805 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:06 smithi016 ceph-mon[31253]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:12:06.806 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:06 smithi016 ceph-mon[31253]: pgmap v685: 65 pgs: 65 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:06 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:06 smithi018 ceph-mon[34587]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:12:06.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:06 smithi018 ceph-mon[34587]: pgmap v685: 65 pgs: 65 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:06.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:06 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/2886405154' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:06.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:06 smithi123 ceph-mon[34645]: osdmap e91: 12 total, 12 up, 12 in 2024-09-21T03:12:06.971 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:06 smithi123 ceph-mon[34645]: pgmap v685: 65 pgs: 65 active+clean; 40 GiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 -- 172.21.15.16:0/1461224873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 msgr2=0x7fcc0c110320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 --2- 172.21.15.16:0/1461224873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c110320 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fcc04009ca0 tx=0x7fcc0402a8c0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 -- 172.21.15.16:0/1461224873 shutdown_connections 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 --2- 172.21.15.16:0/1461224873 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c110320 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 --2- 172.21.15.16:0/1461224873 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c105290 0x7fcc0c10d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 --2- 172.21.15.16:0/1461224873 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c104860 0x7fcc0c104c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 -- 172.21.15.16:0/1461224873 >> 172.21.15.16:0/1461224873 conn(0x7fcc0c079740 msgr2=0x7fcc0c079b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 -- 172.21.15.16:0/1461224873 shutdown_connections 2024-09-21T03:12:07.061 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.059+0000 7fcc127a0640 1 -- 172.21.15.16:0/1461224873 wait complete. 2024-09-21T03:12:07.063 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.060+0000 7fcc127a0640 1 Processor -- start 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.060+0000 7fcc127a0640 1 -- start start 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.060+0000 7fcc127a0640 1 --2- >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c104860 0x7fcc0c19a4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.060+0000 7fcc127a0640 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 0x7fcc0c19aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.060+0000 7fcc127a0640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc127a0640 1 -- --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c113ab0 con 0x7fcc0c10de10 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc127a0640 1 -- --> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c113930 con 0x7fcc0c104860 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc127a0640 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 -- 0x7fcc0c113c30 con 0x7fcc0c105290 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 --2- >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.16:3300/0 says I am v2:172.21.15.16:53034/0 (socket says 172.21.15.16:53034) 2024-09-21T03:12:07.064 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 -- 172.21.15.16:0/3545287130 learned_addr learned my addr 172.21.15.16:0/3545287130 (peer_addr_for_me v2:172.21.15.16:0/0) 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc0b7fe640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 0x7fcc0c19aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc0bfff640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c104860 0x7fcc0c19a4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 -- 172.21.15.16:0/3545287130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 msgr2=0x7fcc0c19aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 0x7fcc0c19aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 -- 172.21.15.16:0/3545287130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c104860 msgr2=0x7fcc0c19a4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c104860 0x7fcc0c19a4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 -- 172.21.15.16:0/3545287130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc0c1a0410 con 0x7fcc0c10de10 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc0b7fe640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 0x7fcc0c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc10d3d640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcc04005cc0 tx=0x7fcc04031700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.061+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc04003b70 con 0x7fcc0c10de10 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.062+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc040318f0 con 0x7fcc0c10de10 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.062+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc0402f950 con 0x7fcc0c10de10 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.062+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc0c1a06a0 con 0x7fcc0c10de10 2024-09-21T03:12:07.065 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.062+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc0c1a6e60 con 0x7fcc0c10de10 2024-09-21T03:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.063+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc0c06c1e0 con 0x7fcc0c10de10 2024-09-21T03:12:07.066 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.065+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 4 ==== mgrmap(e 24) ==== 154643+0+0 (secure 0 0 0) 0x7fcc04004b40 con 0x7fcc0c10de10 2024-09-21T03:12:07.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.066+0000 7fcc097fa640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 0x7fcbf40b8b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-09-21T03:12:07.067 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.066+0000 7fcc0bfff640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 0x7fcbf40b8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-09-21T03:12:07.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.066+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 8016+0+0 (secure 0 0 0) 0x7fcc04108740 con 0x7fcc0c10de10 2024-09-21T03:12:07.068 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.066+0000 7fcc0bfff640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 0x7fcbf40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06c8e0 tx=0x7fcbfc005f90 comp rx=0 tx=0).ready entity=mgr.14226 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-09-21T03:12:07.071 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.069+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+198400 (secure 0 0 0) 0x7fcc040ceee0 con 0x7fcc0c10de10 2024-09-21T03:12:07.272 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.271+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 --> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7fcc0c1a7060 con 0x7fcc0c10de10 2024-09-21T03:12:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.555+0000 7fcc097fa640 1 -- 172.21.15.16:0/3545287130 <== mon.0 v2:172.21.15.16:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v92) ==== 169+0+0 (secure 0 0 0) 0x7fcc040d32c0 con 0x7fcc0c10de10 2024-09-21T03:12:07.557 INFO:teuthology.orchestra.run.smithi016.stderr:pool 'cephfs_data' removed 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 msgr2=0x7fcbf40b8b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 0x7fcbf40b8b90 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fcc0c06c8e0 tx=0x7fcbfc005f90 comp rx=0 tx=0).stop 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 msgr2=0x7fcc0c19fce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7fcc04005cc0 tx=0x7fcc04031700 comp rx=0 tx=0).stop 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 shutdown_connections 2024-09-21T03:12:07.560 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:6800/3994309618,v1:172.21.15.16:6801/3994309618] conn(0x7fcbf40b6680 0x7fcbf40b8b90 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.16:3300/0,v1:172.21.15.16:6789/0] conn(0x7fcc0c10de10 0x7fcc0c19fce0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc0c105290 0x7fcc0c19aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 --2- 172.21.15.16:0/3545287130 >> [v2:172.21.15.18:3300/0,v1:172.21.15.18:6789/0] conn(0x7fcc0c104860 0x7fcc0c19a4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 >> 172.21.15.16:0/3545287130 conn(0x7fcc0c079740 msgr2=0x7fcc0c0ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 shutdown_connections 2024-09-21T03:12:07.561 INFO:teuthology.orchestra.run.smithi016.stderr:2024-09-21T03:12:07.558+0000 7fcc127a0640 1 -- 172.21.15.16:0/3545287130 wait complete. 2024-09-21T03:12:07.928 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:07 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:07.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:07 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:07.970 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:07 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd={"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} : dispatch 2024-09-21T03:12:08.251 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:12:08.278 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:12:08.313 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:12:08.349 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-09-21T03:12:08.349 DEBUG:teuthology.orchestra.run.smithi016:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:12:08.381 DEBUG:teuthology.orchestra.run.smithi018:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:12:08.404 DEBUG:teuthology.orchestra.run.smithi123:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-09-21T03:12:08.425 INFO:tasks.cephadm:Stopping all daemons... 2024-09-21T03:12:08.425 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-09-21T03:12:08.425 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a 2024-09-21T03:12:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:08 smithi123 ceph-mon[34645]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:08.720 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:08 smithi123 ceph-mon[34645]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:12:08.721 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:08 smithi123 ceph-mon[34645]: pgmap v687: 1 pgs: 1 active+clean; 577 KiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:08.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 systemd[1]: Stopping Ceph mon.a for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:08.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 ceph-mon[31253]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:08.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 ceph-mon[31253]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:12:08.767 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 ceph-mon[31253]: pgmap v687: 1 pgs: 1 active+clean; 577 KiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:08 smithi018 ceph-mon[34587]: from='client.? 172.21.15.16:0/3545287130' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-09-21T03:12:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:08 smithi018 ceph-mon[34587]: osdmap e92: 12 total, 12 up, 12 in 2024-09-21T03:12:08.932 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:08 smithi018 ceph-mon[34587]: pgmap v687: 1 pgs: 1 active+clean; 577 KiB data, 126 GiB used, 947 GiB / 1.0 TiB avail 2024-09-21T03:12:09.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a[31249]: 2024-09-21T03:12:08.941+0000 7f777e56c640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:09.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a[31249]: 2024-09-21T03:12:08.941+0000 7f777e56c640 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-09-21T03:12:09.266 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:09 smithi016 podman[97344]: 2024-09-21 03:12:09.126056318 +0000 UTC m=+0.350647057 container died 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:12:09.765 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:09 smithi016 podman[97344]: 2024-09-21 03:12:09.632527153 +0000 UTC m=+0.857117889 container cleanup 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2) 2024-09-21T03:12:09.766 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:09 smithi016 bash[97344]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a 2024-09-21T03:12:10.187 INFO:journalctl@ceph.mon.a.smithi016.stdout:Sep 21 03:12:09 smithi016 podman[97356]: 2024-09-21 03:12:09.918326081 +0000 UTC m=+0.786907820 container remove 9f2db1d0d395d44ddbe3abf114cfce3d2a0ec67a344a745196c3f119ffdd772a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-a, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:12:10.202 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.a.service' 2024-09-21T03:12:10.260 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:10.260 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-09-21T03:12:10.261 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-09-21T03:12:10.261 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.b 2024-09-21T03:12:10.683 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:10 smithi018 systemd[1]: Stopping Ceph mon.b for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b[34583]: 2024-09-21T03:12:10.740+0000 7fc7d3a0d640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:11.182 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b[34583]: 2024-09-21T03:12:10.740+0000 7fc7d3a0d640 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:12:11.183 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:10 smithi018 podman[66570]: 2024-09-21 03:12:10.825000435 +0000 UTC m=+0.227625454 container died da9c74c5bb839abcbed47ac5409d19cf1f230436131dd1dd6aaecfe3c50449d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:12:11.519 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 podman[66570]: 2024-09-21 03:12:11.233292847 +0000 UTC m=+0.635917869 container cleanup da9c74c5bb839abcbed47ac5409d19cf1f230436131dd1dd6aaecfe3c50449d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b, CEPH_POINT_RELEASE=, ceph=True, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:12:11.519 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 bash[66570]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b 2024-09-21T03:12:11.755 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.b.service' 2024-09-21T03:12:11.784 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 podman[66580]: 2024-09-21 03:12:11.518442822 +0000 UTC m=+0.684124080 container remove da9c74c5bb839abcbed47ac5409d19cf1f230436131dd1dd6aaecfe3c50449d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-b, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-09-21T03:12:11.784 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.b.service: Deactivated successfully. 2024-09-21T03:12:11.784 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 systemd[1]: Stopped Ceph mon.b for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T03:12:11.784 INFO:journalctl@ceph.mon.b.smithi018.stdout:Sep 21 03:12:11 smithi018 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.b.service: Consumed 13.563s CPU time. 2024-09-21T03:12:11.842 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:11.842 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-09-21T03:12:11.842 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-09-21T03:12:11.842 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.c 2024-09-21T03:12:12.202 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:11 smithi123 systemd[1]: Stopping Ceph mon.c for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:12.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c[34630]: 2024-09-21T03:12:12.200+0000 7f995fddc640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:12.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c[34630]: 2024-09-21T03:12:12.200+0000 7f995fddc640 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-09-21T03:12:12.471 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:12 smithi123 podman[67908]: 2024-09-21 03:12:12.292304117 +0000 UTC m=+0.255413792 container died 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:12:12.944 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:12 smithi123 podman[67908]: 2024-09-21 03:12:12.675315555 +0000 UTC m=+0.638425231 container cleanup 4bfb0da369426db6dec5f4feb9b29d99b3fa9779bb239ed40b477dcaf8c458e5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2) 2024-09-21T03:12:12.944 INFO:journalctl@ceph.mon.c.smithi123.stdout:Sep 21 03:12:12 smithi123 bash[67908]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mon-c 2024-09-21T03:12:13.118 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mon.c.service' 2024-09-21T03:12:13.165 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:13.165 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-09-21T03:12:13.165 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-09-21T03:12:13.165 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x 2024-09-21T03:12:13.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:13 smithi016 systemd[1]: Stopping Ceph mgr.x for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:13.883 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x[31481]: 2024-09-21T03:12:13.564+0000 7f502d59e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:13.883 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:13 smithi016 podman[97456]: 2024-09-21 03:12:13.605493351 +0000 UTC m=+0.184273204 container died 136364bf88ff729dbcc607c04c160d9a3afadc063d0597453945c6f03a1b4c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:12:14.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:14 smithi016 podman[97456]: 2024-09-21 03:12:14.139968097 +0000 UTC m=+0.718747945 container cleanup 136364bf88ff729dbcc607c04c160d9a3afadc063d0597453945c6f03a1b4c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:12:14.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:14 smithi016 bash[97456]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x 2024-09-21T03:12:14.516 INFO:journalctl@ceph.mgr.x.smithi016.stdout:Sep 21 03:12:14 smithi016 podman[97466]: 2024-09-21 03:12:14.358642717 +0000 UTC m=+0.754979211 container remove 136364bf88ff729dbcc607c04c160d9a3afadc063d0597453945c6f03a1b4c46 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:1843221895adccfb3b692060251dd6a0183bc410, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-x, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-09-21T03:12:14.697 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.x.service' 2024-09-21T03:12:14.736 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:14.736 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-09-21T03:12:14.736 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-09-21T03:12:14.736 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.y 2024-09-21T03:12:15.102 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 03:12:14 smithi018 systemd[1]: Stopping Ceph mgr.y for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:15.390 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 03:12:15 smithi018 podman[66693]: 2024-09-21 03:12:15.101553445 +0000 UTC m=+0.138106849 container died fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:12:15.682 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 03:12:15 smithi018 podman[66693]: 2024-09-21 03:12:15.497049538 +0000 UTC m=+0.533602924 container cleanup fce0394e33363a8dae17a8dd56f648181cc5990100f969924c62d359e1a8361b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2) 2024-09-21T03:12:15.682 INFO:journalctl@ceph.mgr.y.smithi018.stdout:Sep 21 03:12:15 smithi018 bash[66693]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-y 2024-09-21T03:12:15.878 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.y.service' 2024-09-21T03:12:15.914 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:15.914 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-09-21T03:12:15.914 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-09-21T03:12:15.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.z 2024-09-21T03:12:16.220 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 21 03:12:15 smithi123 systemd[1]: Stopping Ceph mgr.z for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:16.515 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 21 03:12:16 smithi123 podman[68027]: 2024-09-21 03:12:16.242992852 +0000 UTC m=+0.137000069 container died 20b935b1def7bac9c378e56e5e2b97732dfea550d28359fdefd9e20729c9e50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-z, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, ceph=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:12:16.873 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 21 03:12:16 smithi123 podman[68027]: 2024-09-21 03:12:16.621783781 +0000 UTC m=+0.515790998 container cleanup 20b935b1def7bac9c378e56e5e2b97732dfea550d28359fdefd9e20729c9e50f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-z, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:12:16.874 INFO:journalctl@ceph.mgr.z.smithi123.stdout:Sep 21 03:12:16 smithi123 bash[68027]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mgr-z 2024-09-21T03:12:17.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mgr.z.service' 2024-09-21T03:12:17.086 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:17.086 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-09-21T03:12:17.086 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-09-21T03:12:17.087 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.0 2024-09-21T03:12:17.379 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:17 smithi016 systemd[1]: Stopping Ceph osd.0 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:17.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T03:12:17.377+0000 7f55a8dd7640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:17.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T03:12:17.377+0000 7f55a8dd7640 -1 osd.0 92 *** Got signal Terminated *** 2024-09-21T03:12:17.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0[45711]: 2024-09-21T03:12:17.377+0000 7f55a8dd7640 -1 osd.0 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:12:23.363 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:23 smithi016 podman[97571]: 2024-09-21 03:12:23.084161576 +0000 UTC m=+5.809074676 container died 0a0eacf41db0c26fa4332c1eb12dd6eeb61bbcf9b02ebc2bef64e37e87d10582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:12:23.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:23 smithi016 podman[97571]: 2024-09-21 03:12:23.545128245 +0000 UTC m=+6.270041374 container cleanup 0a0eacf41db0c26fa4332c1eb12dd6eeb61bbcf9b02ebc2bef64e37e87d10582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:12:23.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:23 smithi016 bash[97571]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0 2024-09-21T03:12:24.156 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:23 smithi016 podman[97586]: 2024-09-21 03:12:23.830165478 +0000 UTC m=+0.740358257 container remove 0a0eacf41db0c26fa4332c1eb12dd6eeb61bbcf9b02ebc2bef64e37e87d10582 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, io.buildah.version=1.37.2) 2024-09-21T03:12:24.409 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.156024575 +0000 UTC m=+0.097585381 container create 59656b6eb65e18b8c869defe77d597fd04f759a7238afe1be631fbfc68d78e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-09-21T03:12:24.409 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.074896534 +0000 UTC m=+0.016457322 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:12:24.409 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.408470685 +0000 UTC m=+0.350031469 container init 59656b6eb65e18b8c869defe77d597fd04f759a7238afe1be631fbfc68d78e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:12:24.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.412081217 +0000 UTC m=+0.353641982 container start 59656b6eb65e18b8c869defe77d597fd04f759a7238afe1be631fbfc68d78e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2) 2024-09-21T03:12:24.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.500456337 +0000 UTC m=+0.442017116 container attach 59656b6eb65e18b8c869defe77d597fd04f759a7238afe1be631fbfc68d78e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0-deactivate, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:24.766 INFO:journalctl@ceph.osd.0.smithi016.stdout:Sep 21 03:12:24 smithi016 podman[97676]: 2024-09-21 03:12:24.609720466 +0000 UTC m=+0.551281239 container died 59656b6eb65e18b8c869defe77d597fd04f759a7238afe1be631fbfc68d78e43 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-0-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:12:25.207 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.0.service' 2024-09-21T03:12:25.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:25.251 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-09-21T03:12:25.251 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-09-21T03:12:25.251 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.1 2024-09-21T03:12:25.561 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:25 smithi018 systemd[1]: Stopping Ceph osd.1 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:25.932 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T03:12:25.559+0000 7f4417cb6640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:25.932 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T03:12:25.559+0000 7f4417cb6640 -1 osd.1 92 *** Got signal Terminated *** 2024-09-21T03:12:25.932 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1[39281]: 2024-09-21T03:12:25.559+0000 7f4417cb6640 -1 osd.1 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:12:31.385 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:31 smithi018 podman[66820]: 2024-09-21 03:12:31.133240508 +0000 UTC m=+5.685829007 container died 594cda2bfe8a57cfe12bf509bf93c6620f8ed377342783808f53974beacf5d73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS) 2024-09-21T03:12:31.682 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:31 smithi018 podman[66820]: 2024-09-21 03:12:31.467091358 +0000 UTC m=+6.019679858 container cleanup 594cda2bfe8a57cfe12bf509bf93c6620f8ed377342783808f53974beacf5d73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:12:31.682 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:31 smithi018 bash[66820]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1 2024-09-21T03:12:32.133 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:31 smithi018 podman[66836]: 2024-09-21 03:12:31.685172268 +0000 UTC m=+0.549578776 container remove 594cda2bfe8a57cfe12bf509bf93c6620f8ed377342783808f53974beacf5d73 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True) 2024-09-21T03:12:32.397 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.034913024 +0000 UTC m=+0.016141784 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:12:32.398 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.136009987 +0000 UTC m=+0.117238735 container create dcbed926a988461753668b37fec706c8ccea7df320691090b66d3414eb682398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2) 2024-09-21T03:12:32.398 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.313502087 +0000 UTC m=+0.294730832 container init dcbed926a988461753668b37fec706c8ccea7df320691090b66d3414eb682398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:12:32.398 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.316766294 +0000 UTC m=+0.297995046 container start dcbed926a988461753668b37fec706c8ccea7df320691090b66d3414eb682398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1-deactivate, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0) 2024-09-21T03:12:32.682 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.396471848 +0000 UTC m=+0.377700603 container attach dcbed926a988461753668b37fec706c8ccea7df320691090b66d3414eb682398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1-deactivate, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:32.683 INFO:journalctl@ceph.osd.1.smithi018.stdout:Sep 21 03:12:32 smithi018 podman[66923]: 2024-09-21 03:12:32.508636676 +0000 UTC m=+0.489865429 container died dcbed926a988461753668b37fec706c8ccea7df320691090b66d3414eb682398 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-1-deactivate, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:12:33.095 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.1.service' 2024-09-21T03:12:33.136 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:33.136 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-09-21T03:12:33.136 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-09-21T03:12:33.136 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.2 2024-09-21T03:12:33.435 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:33 smithi123 systemd[1]: Stopping Ceph osd.2 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:33.720 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T03:12:33.432+0000 7f0f67c01640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:33.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T03:12:33.432+0000 7f0f67c01640 -1 osd.2 92 *** Got signal Terminated *** 2024-09-21T03:12:33.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2[39743]: 2024-09-21T03:12:33.432+0000 7f0f67c01640 -1 osd.2 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:12:44.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:44.343+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:45.338 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:45 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:45.038+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:45.338 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:45 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:45.240+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:45.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:45.011+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:45.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:45.325+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:46.349 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:46 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:46.038+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:46.350 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:46 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:46.253+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:46.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:46.016+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:46.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:46.288+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:47.277 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:47.026+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:47.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:46.980+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:47.637 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:47.384+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:47.637 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:47 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:47.276+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:47.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:47.255+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:47.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:47.393+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:48.377 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:48 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:48.030+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:48.377 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:48 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:48.276+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:48.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:47.999+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:48.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:48.242+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:48.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:48.415+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:48.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:48 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:48.375+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:49.425 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:49 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:49.285+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:49.425 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:49 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:49.076+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:49.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:49.030+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:49.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:49.281+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:49.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:49.392+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:49.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:49 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:49.423+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:50.220 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:49 smithi123 podman[68167]: 2024-09-21 03:12:49.858318238 +0000 UTC m=+16.536721567 container died d60c740190bb576ed24044dd26f452c435150a27dc2cef2da82dbfa7d82d0ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:12:50.393 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:50 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:50.075+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:50.394 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:50 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:50.299+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:50.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:50.075+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:50.609 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:50.249+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:50.609 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:50.412+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:50.610 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:50 smithi123 podman[68167]: 2024-09-21 03:12:50.357355771 +0000 UTC m=+17.035759096 container cleanup d60c740190bb576ed24044dd26f452c435150a27dc2cef2da82dbfa7d82d0ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:12:50.610 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:50 smithi123 bash[68167]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2 2024-09-21T03:12:50.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:50 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:50.392+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:50.966 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:50 smithi123 podman[68182]: 2024-09-21 03:12:50.608611256 +0000 UTC m=+0.744907513 container remove d60c740190bb576ed24044dd26f452c435150a27dc2cef2da82dbfa7d82d0ab1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:51.220 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:50 smithi123 podman[68272]: 2024-09-21 03:12:50.868704294 +0000 UTC m=+0.016094769 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:12:51.221 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:51 smithi123 podman[68272]: 2024-09-21 03:12:51.012920703 +0000 UTC m=+0.160311221 container create 3e38cf64048666444f94a299a82f6d50a2b35a8f3ba0a263367800e6577013f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0) 2024-09-21T03:12:51.419 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:51 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:51.066+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:51.420 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:51 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:51.298+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:51.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:51.067+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:51.720 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:51.220+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:51.409+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:51.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:51 smithi123 podman[68272]: 2024-09-21 03:12:51.253992755 +0000 UTC m=+0.401383231 container init 3e38cf64048666444f94a299a82f6d50a2b35a8f3ba0a263367800e6577013f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2-deactivate, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:12:51.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:51 smithi123 podman[68272]: 2024-09-21 03:12:51.257340141 +0000 UTC m=+0.404730616 container start 3e38cf64048666444f94a299a82f6d50a2b35a8f3ba0a263367800e6577013f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:12:51.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:51 smithi123 podman[68272]: 2024-09-21 03:12:51.387330316 +0000 UTC m=+0.534720808 container attach 3e38cf64048666444f94a299a82f6d50a2b35a8f3ba0a263367800e6577013f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:51.721 INFO:journalctl@ceph.osd.2.smithi123.stdout:Sep 21 03:12:51 smithi123 podman[68272]: 2024-09-21 03:12:51.458065797 +0000 UTC m=+0.605456286 container died 3e38cf64048666444f94a299a82f6d50a2b35a8f3ba0a263367800e6577013f9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-2-deactivate, ceph=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:51.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:51 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:51.418+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:52.151 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.2.service' 2024-09-21T03:12:52.190 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:12:52.190 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-09-21T03:12:52.190 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-09-21T03:12:52.190 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.3 2024-09-21T03:12:52.370 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:52.067+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:52.371 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:52 smithi016 systemd[1]: Stopping Ceph osd.3 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:12:52.371 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:52.278+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:52.421 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:52.195+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:52.421 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:52.370+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:52.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:52.110+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:52.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:52.368+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:52.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:52.368+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:52.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:52.497+0000 7f55d832b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:12:52.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:52.497+0000 7f55d832b640 -1 osd.3 92 *** Got signal Terminated *** 2024-09-21T03:12:52.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:52 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:52.497+0000 7f55d832b640 -1 osd.3 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:12:53.176 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:52.886+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:53.391 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:53 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:53.090+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:53.391 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:53 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:53.090+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.994732+0000 front 2024-09-21T03:12:28.994812+0000 (oldest deadline 2024-09-21T03:12:52.494649+0000) 2024-09-21T03:12:53.391 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:53 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:53.231+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:53.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:53 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:53.078+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:53.448 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:53.174+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:53.448 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:53.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:53.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:53 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:53.389+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:53.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:53 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:53.389+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:54.129 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:53.843+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:54.349 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:54 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:54.237+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:54.350 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:54 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:54.075+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:54.350 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:54 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:54.075+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.994732+0000 front 2024-09-21T03:12:28.994812+0000 (oldest deadline 2024-09-21T03:12:52.494649+0000) 2024-09-21T03:12:54.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:54 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:54.109+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:54.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:54 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:54.109+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:54.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:54.316+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:54.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:54.128+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:54.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:54.128+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:54.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:54 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:54.348+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:54.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:54 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:54.348+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:55.132 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:54.809+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:55.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:55 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:55.103+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:55.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:55 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:55.103+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:55.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:55.130+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:55.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:55.130+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:55.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:55.293+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:55.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:55.293+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:12:55.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:55.228+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:55.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:55.333+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:55.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:55.333+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:55.516 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:55.097+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:55.516 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:55 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:55.097+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.994732+0000 front 2024-09-21T03:12:28.994812+0000 (oldest deadline 2024-09-21T03:12:52.494649+0000) 2024-09-21T03:12:56.070 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:12:55 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:12:55.764+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:12:56.121 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:55.810+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:56.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:56 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:56.068+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:56.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:56 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:56.068+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:56.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:56.120+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:56.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:56.120+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:56.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:56.267+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:56.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:56.267+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:12:56.515 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:56.263+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:56.516 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:56.125+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:56.516 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:56.125+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.994732+0000 front 2024-09-21T03:12:28.994812+0000 (oldest deadline 2024-09-21T03:12:52.494649+0000) 2024-09-21T03:12:56.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:56.350+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:56.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:56 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:56.350+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:57.071 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:12:56 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:12:56.773+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:12:57.146 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:56.764+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:57.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:12:57 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:12:57.094+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:12:57.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:57 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:57.069+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:57.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:57 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:57.069+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:57.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:57.144+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:57.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:57.144+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:57.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:57.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:57.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:57.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:12:57.515 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:57.090+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.092470+0000 front 2024-09-21T03:12:22.092768+0000 (oldest deadline 2024-09-21T03:12:44.392765+0000) 2024-09-21T03:12:57.515 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3[52139]: 2024-09-21T03:12:57.090+0000 7f55d4916640 -1 osd.3 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.994732+0000 front 2024-09-21T03:12:28.994812+0000 (oldest deadline 2024-09-21T03:12:52.494649+0000) 2024-09-21T03:12:57.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:57.221+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:57.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:57.221+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:12:57.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:57.306+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:57.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:57 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:57.306+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:58.061 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:12:57 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:12:57.726+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:12:58.101 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:57.809+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:58.323 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:58 smithi016 podman[97791]: 2024-09-21 03:12:58.070582923 +0000 UTC m=+5.677909206 container died 5868d3195157fcec98915bbdd57e199d97065db89e3b77461bbe62e6c409550c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2) 2024-09-21T03:12:58.323 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:58 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:58.181+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:58.323 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:58 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:58.181+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:12:58.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:12:58 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:12:58.121+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:12:58.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:58 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:58.060+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:58.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:58 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:58.060+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:58.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:58.100+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:58.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:58.100+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:58.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:58.278+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:58.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:58.278+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:12:58.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:58 smithi016 podman[97791]: 2024-09-21 03:12:58.523654865 +0000 UTC m=+6.130981215 container cleanup 5868d3195157fcec98915bbdd57e199d97065db89e3b77461bbe62e6c409550c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:12:58.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:58 smithi016 bash[97791]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3 2024-09-21T03:12:58.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:58 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:58.321+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:58.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:58 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:58.321+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:59.062 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:58.764+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:12:59.109 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:12:58 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:12:58.685+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:12:59.146 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:58 smithi016 podman[97806]: 2024-09-21 03:12:58.800082182 +0000 UTC m=+0.724487378 container remove 5868d3195157fcec98915bbdd57e199d97065db89e3b77461bbe62e6c409550c (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:12:59.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:12:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:12:59.126+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:12:59.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:59.108+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:12:59.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:12:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:12:59.108+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:12:59.445 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:59 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:59.340+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:12:59.445 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:12:59 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:12:59.340+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:12:59.446 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:59 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:59.199+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:12:59.446 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:12:59 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:12:59.199+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:12:59.446 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.04895485 +0000 UTC m=+0.016253314 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:12:59.446 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.196598292 +0000 UTC m=+0.163896732 container create 8651917e8238ca7eae5ac61d703788d2e9ccf693e8a03a2ded6853aa7da544e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2) 2024-09-21T03:12:59.446 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.445182522 +0000 UTC m=+0.412480974 container init 8651917e8238ca7eae5ac61d703788d2e9ccf693e8a03a2ded6853aa7da544e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:12:59.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:59.061+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:12:59.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:12:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:12:59.061+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:12:59.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:59.250+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:12:59.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:12:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:12:59.250+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:12:59.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.448877605 +0000 UTC m=+0.416176051 container start 8651917e8238ca7eae5ac61d703788d2e9ccf693e8a03a2ded6853aa7da544e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:12:59.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.528505822 +0000 UTC m=+0.495804269 container attach 8651917e8238ca7eae5ac61d703788d2e9ccf693e8a03a2ded6853aa7da544e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:12:59.766 INFO:journalctl@ceph.osd.3.smithi016.stdout:Sep 21 03:12:59 smithi016 podman[97896]: 2024-09-21 03:12:59.640515101 +0000 UTC m=+0.607813552 container died 8651917e8238ca7eae5ac61d703788d2e9ccf693e8a03a2ded6853aa7da544e9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-3-deactivate, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-09-21T03:13:00.091 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:12:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:12:59.738+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:00.100 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:12:59 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:12:59.728+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:00.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:00.198+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:00.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:00.198+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:00.339 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.3.service' 2024-09-21T03:13:00.388 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:00.388 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-09-21T03:13:00.388 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-09-21T03:13:00.388 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.4 2024-09-21T03:13:00.416 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:00.098+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:00.416 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:00.100+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:00.416 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:00.100+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:00.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:00.089+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:00.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:00.089+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:00.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:00.228+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:00.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:00.228+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:00.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:00.362+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:00.610 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:00 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:00.362+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:00.682 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:00 smithi018 systemd[1]: Stopping Ceph osd.4 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:01.054 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:00.684+0000 7ff93e014640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:01.055 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:00.684+0000 7ff93e014640 -1 osd.4 92 *** Got signal Terminated *** 2024-09-21T03:13:01.055 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:00.684+0000 7ff93e014640 -1 osd.4 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:01.055 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:00 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:00.713+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:01.094 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:00.740+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:01.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:01.086+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:01.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:01.053+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:01.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:01.053+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:01.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:01.092+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:01.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:01.092+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:01.197+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:01.197+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:01.515 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:01.375+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:01.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:01.375+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:01.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:01.206+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:01.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:01 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:01.206+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:01.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:01.712+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:02.015 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:01 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:01.693+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:02.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:02 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:02.131+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:02.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:02 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:02.014+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:02.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:02 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:02.014+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:02.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:02 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:02.055+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:02.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:02 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:02.056+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:02.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:02 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:02.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:02.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:02 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:02.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:02.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:02 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:02.255+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:02.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:02 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:02.255+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:02.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:02 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:02.399+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:02.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:02 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:02.399+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:03.004 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:02 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:02.692+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:03.023 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:02 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:02.757+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:03.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:03 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:03.168+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:03.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:03 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:03.003+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:03.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:03 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:03.003+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:03.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:03 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:03.021+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:03.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:03 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:03.021+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:03 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:03.167+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:03 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:03.167+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:03.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:03 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:03.256+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:03.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:03 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:03.256+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:03.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:03 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:03.439+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:03.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:03 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:03.439+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:04.044 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:03 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:03.690+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:04.132 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:03 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:03.774+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:04.132 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:04.006+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:04.132 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:04.006+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:04.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:04 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:04.194+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:04.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:04 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:04.042+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:04.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:04 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:04.042+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:04.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:04.130+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:04.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:04.130+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:04.766 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:04.280+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:04.766 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:04.280+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:04.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:04.412+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:04.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:04.412+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:05.013 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:04 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4[45442]: 2024-09-21T03:13:04.727+0000 7ff9365f7640 -1 osd.4 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:25.131039+0000 front 2024-09-21T03:12:25.131068+0000 (oldest deadline 2024-09-21T03:12:55.131859+0000) 2024-09-21T03:13:05.036 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:04 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:04.731+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:05.432 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:05 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:05.194+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:05.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:05 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:05.011+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:05.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:05 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:05.011+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:05.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:05.034+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:05.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:05.034+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:05.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:05.164+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:05.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:05.164+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:05.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:05 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:05.260+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:05.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:05 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:05.260+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:05.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:05 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:05.381+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:05.517 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:05 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:05.381+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:05.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:05.711+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:06.237 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:06 smithi018 podman[67037]: 2024-09-21 03:13:06.191167547 +0000 UTC m=+5.607991435 container died 3a7d10f0f3f97999624a1ac6840104c321b0109f56ca97ef6f314a092e74144d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:13:06.238 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:05 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:05.971+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:06.238 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:05 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:05.971+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:06.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:06 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:06.130+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:06.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:06 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:06.130+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:06.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:05.996+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:06.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:05 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:05.996+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:06.509 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:06 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:06.235+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:06.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:06 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:06.233+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:06.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:06 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:06.233+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:06.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:06 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:06.411+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:06.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:06 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:06.411+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:06.932 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:06 smithi018 podman[67037]: 2024-09-21 03:13:06.508925179 +0000 UTC m=+5.925749101 container cleanup 3a7d10f0f3f97999624a1ac6840104c321b0109f56ca97ef6f314a092e74144d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD) 2024-09-21T03:13:06.932 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:06 smithi018 bash[67037]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4 2024-09-21T03:13:06.932 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:06 smithi018 podman[67053]: 2024-09-21 03:13:06.718332896 +0000 UTC m=+0.526269736 container remove 3a7d10f0f3f97999624a1ac6840104c321b0109f56ca97ef6f314a092e74144d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD) 2024-09-21T03:13:06.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:06 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:06.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:07.230 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.110732057 +0000 UTC m=+0.100597498 container create 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-09-21T03:13:07.230 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.026293936 +0000 UTC m=+0.016159345 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:07.230 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:06 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:06.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:07.230 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:06 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:06.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:07.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:07.009+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:07.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:07.009+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:07.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:07.181+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:07.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:07.181+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:07.515 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:07 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:07.259+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:07.515 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:07 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:07.259+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:07.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:07 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:07.435+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:07.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:07 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:07.435+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:07.521 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:07 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:07.233+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:07.522 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.329731679 +0000 UTC m=+0.319597080 container init 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, ceph=True, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-09-21T03:13:07.522 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.333090969 +0000 UTC m=+0.322956372 container start 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-09-21T03:13:07.522 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.429775196 +0000 UTC m=+0.419640608 container attach 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:07.826 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:07 smithi018 podman[67140]: 2024-09-21 03:13:07.521110815 +0000 UTC m=+0.510976215 container died 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:13:07.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:07.702+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:08.079 INFO:journalctl@ceph.osd.4.smithi018.stdout:Sep 21 03:13:08 smithi018 podman[67164]: 2024-09-21 03:13:08.058444479 +0000 UTC m=+0.525754208 container remove 7fdcacede146e04c8d3ad92e334a0cc61982316963edcec94de939d360cdb0ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-4-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:08.079 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:07 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:07.912+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:08.080 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:07 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:07.912+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:08.084 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.4.service' 2024-09-21T03:13:08.163 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:08.163 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-09-21T03:13:08.163 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-09-21T03:13:08.163 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.5 2024-09-21T03:13:08.342 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:08 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:08.275+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:08.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:08.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:08.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:08.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:08.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:07.977+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:08.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:07 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:07.977+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:08.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 systemd[1]: Stopping Ceph osd.5 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:08.515 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:08.236+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:08.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:08.236+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:08.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:08.401+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:08.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:08.401+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:08.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:08.684+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:08.971 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:08.634+0000 7fde34ac8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:08.971 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:08.634+0000 7fde34ac8640 -1 osd.5 92 *** Got signal Terminated *** 2024-09-21T03:13:08.971 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:08.634+0000 7fde34ac8640 -1 osd.5 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:09.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:08 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:08.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:09.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:08 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:08.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:09.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:08.985+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:09.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:08 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:08.985+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:09.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:09 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:09.219+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:09.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:09 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:09.219+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:09.515 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:09 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:09.196+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:09.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:09 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:09.196+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:09.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:09 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:09.434+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:09.516 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:09 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:09.434+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:09.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:09 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:09.303+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:10.009 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:09 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:09.728+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:10.300 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:09 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:09.949+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:10.300 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:09 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:09.949+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:10.419 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:10 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:10.160+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:10.419 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:10 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:10.160+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:10.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:10.006+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:10.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:10.006+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:10.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:10.178+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:10.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:10.178+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:10.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:10.299+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:10.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:10 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:10.417+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:10.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:10 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:10.417+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:10.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:10.705+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:11.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:10.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:11.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:10.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:11.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:10 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:10.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:11.444 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:11 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:11.128+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:11.444 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:11 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:11.128+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:11.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:11.212+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:11.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:11.212+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:11.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:10.971+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:11.471 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:10 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:10.971+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:11.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:11 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:11.322+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:11.706 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:11 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:11.443+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:11.707 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:11 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:11.443+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:11.970 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:11.924+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:11.970 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:11.924+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:11.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:11.697+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:11.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:11 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:11.697+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:12.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:11 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:11.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:12.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:11 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:11.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:12.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:11 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:11.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:12.455 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:12 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:12.080+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:12.455 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:12 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:12.080+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:12.455 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:12 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:12.080+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:12.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:12.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:12.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:12.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:12.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:12.333+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:12.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:12 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:12.454+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:12.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:12 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:12.454+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:12.931 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:12.674+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:12.931 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:12.674+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:13.198 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:12.928+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:20.024032+0000 front 2024-09-21T03:12:20.024045+0000 (oldest deadline 2024-09-21T03:12:43.524072+0000) 2024-09-21T03:13:13.198 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:12.928+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.824926+0000 front 2024-09-21T03:12:28.824912+0000 (oldest deadline 2024-09-21T03:12:53.524906+0000) 2024-09-21T03:13:13.198 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:12 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5[46342]: 2024-09-21T03:13:12.928+0000 7fde2d0ab640 -1 osd.5 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:35.726129+0000 front 2024-09-21T03:12:35.726107+0000 (oldest deadline 2024-09-21T03:13:12.227076+0000) 2024-09-21T03:13:13.298 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:12.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:13.298 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:12.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:13.299 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:12 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:12.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:13.425 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:13.040+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:13.425 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:13.040+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:13.425 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:13.040+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:13.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:13 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:13.195+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:13.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:13 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:13.195+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:13.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:13 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:13.195+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:13.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:13 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:13.297+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:13.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:13.424+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:13.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:13.424+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:13.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:13.424+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:13.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:13 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:13.644+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:13.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:13 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:13.644+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:14.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:13 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:13.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:14.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:13 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:13.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:14.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:13 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:13.905+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:14.448 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.018+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:14.448 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.018+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:14.449 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.018+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:14.470 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:14 smithi123 podman[68385]: 2024-09-21 03:13:14.311031801 +0000 UTC m=+5.963515055 container died 4196fe09e6cfd8bf7035d340b8bf10eabedc110708e279868a2b878ff3ce258b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731) 2024-09-21T03:13:14.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:14 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:14.164+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:14.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:14 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:14.164+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:14.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:14 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:14.164+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:14.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:14 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:14.264+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:14.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:14.447+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:14.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:14.447+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:14.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:14.447+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:14.854 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:14 smithi123 podman[68385]: 2024-09-21 03:13:14.635132943 +0000 UTC m=+6.287616197 container cleanup 4196fe09e6cfd8bf7035d340b8bf10eabedc110708e279868a2b878ff3ce258b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240731, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:13:14.854 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:14 smithi123 bash[68385]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5 2024-09-21T03:13:14.854 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:14 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:14.653+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:14.854 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:14 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:14.653+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:15.119 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:14 smithi123 podman[68400]: 2024-09-21 03:13:14.853040963 +0000 UTC m=+0.536385422 container remove 4196fe09e6cfd8bf7035d340b8bf10eabedc110708e279868a2b878ff3ce258b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:13:15.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:14 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:14.897+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:15.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:14 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:14.897+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:15.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:14 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:14.897+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:15.265 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.982+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:15.265 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.982+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:15.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:14 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:14.982+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:15.397 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.194592837 +0000 UTC m=+0.015670704 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:15.398 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.295500212 +0000 UTC m=+0.116578079 container create 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:13:15.398 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:15 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:15.116+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:15.398 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:15 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:15.116+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:15.398 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:15 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:15.116+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:15.553 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:15 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:15.453+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:15.553 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:15 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:15.453+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:15.553 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:15 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:15.453+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:15.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:15 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:15.241+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:15.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:15 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:15.241+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:15.700 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:15 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:15.604+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:15.700 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:15 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:15.604+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:15.700 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.506219826 +0000 UTC m=+0.327297694 container init 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2) 2024-09-21T03:13:15.700 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.509482054 +0000 UTC m=+0.330559925 container start 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:15.700 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.581040132 +0000 UTC m=+0.402118014 container attach 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:15.970 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:15 smithi123 podman[68491]: 2024-09-21 03:13:15.699922924 +0000 UTC m=+0.521000802 container died 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:13:16.226 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:15 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:15.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:16.226 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:15 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:15.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:16.226 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:15 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:15.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:16.236 INFO:journalctl@ceph.osd.5.smithi123.stdout:Sep 21 03:13:16 smithi123 podman[68514]: 2024-09-21 03:13:16.226120853 +0000 UTC m=+0.516595150 container remove 791375cb19b74051f34e4ac8a4626e5065c464c57b7b1a7d5bbb0ed789ca28fc (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-5-deactivate, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2) 2024-09-21T03:13:16.236 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:16 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:16.121+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:16.236 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:16 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:16.121+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:16.237 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:16 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:16.121+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:16.241 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.5.service' 2024-09-21T03:13:16.265 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.005+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:16.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.005+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:16.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.005+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:16.278 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:16.279 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-09-21T03:13:16.279 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-09-21T03:13:16.279 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.6 2024-09-21T03:13:16.573 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 systemd[1]: Stopping Ceph osd.6 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:16.574 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:16.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:16.574 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:16.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:16.574 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:16.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:16.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:16 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:16.224+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:16.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:16 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:16.224+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:16.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:16 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:16.654+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:16.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:16 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:16.654+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.618+0000 7f0da6405640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.618+0000 7f0da6405640 -1 osd.6 92 *** Got signal Terminated *** 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.618+0000 7f0da6405640 -1 osd.6 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.971+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.971+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:16.973 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:16 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:16.971+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:17.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:16 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:16.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:17.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:16 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:16.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:17.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:16 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:16.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:17.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:17 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:17.147+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:17.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:17 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:17.147+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:17.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:17 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:17.147+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:17.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:17.269+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:17.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:17.269+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:17.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:17.506+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:17.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:17.506+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:17.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:17.506+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:17.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:17 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:17.649+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:17.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:17 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:17.649+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:18.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:17.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:18.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:17.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:18.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:17 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:17.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:18.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:17.940+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:18.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:17.940+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:18.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:17 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:17.940+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:18.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:18 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:18.181+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:18.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:18 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:18.181+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:18.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:18 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:18.181+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:18.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:18 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:18.311+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:18.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:18 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:18.311+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:18.906 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:18.525+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:18.906 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:18.525+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:18.906 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:18.525+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:18.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:18 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:18.677+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:18.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:18 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:18.677+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:19.265 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:18.904+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:19.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:18.904+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:19.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:18.904+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:19.291 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:18 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:18.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:19.292 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:18 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:18.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:19.292 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:18 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:18.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:19.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:19 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:19.143+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:19.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:19 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:19.143+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:19.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:19 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:19.143+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:19.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:19.290+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:19.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:19.290+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:19.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:19.290+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:19.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:19.535+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:19.880 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:19.535+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:19.881 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:19.535+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:19.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:19 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:19.660+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:19.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:19 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:19.661+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:20.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:19.879+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:20.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:19.879+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:20.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:19 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:19.879+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:20.339 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:19.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:20.340 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:19.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:20.340 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:19 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:19.972+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:20.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:20 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:20.159+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:20.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:20 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:20.159+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:20.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:20 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:20.159+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:20.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:20.338+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:20.690 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:20.338+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:20.690 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:20.338+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:20.888 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:20.539+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:20.888 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:20.539+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:20.888 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:20.539+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:20.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:20 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:20.615+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:20.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:20 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:20.615+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:21.265 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:20.887+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:21.878258+0000 front 2024-09-21T03:12:21.878347+0000 (oldest deadline 2024-09-21T03:12:44.778408+0000) 2024-09-21T03:13:21.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:20.887+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.980573+0000 front 2024-09-21T03:12:29.980511+0000 (oldest deadline 2024-09-21T03:12:56.981182+0000) 2024-09-21T03:13:21.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:20 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6[58266]: 2024-09-21T03:13:20.887+0000 7f0d9e9e8640 -1 osd.6 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.981667+0000 front 2024-09-21T03:12:36.981692+0000 (oldest deadline 2024-09-21T03:13:11.583394+0000) 2024-09-21T03:13:21.373 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:20.957+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:21.373 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:20.957+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:21.373 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:20.957+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:21.373 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:20 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:20.957+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:21.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:21.165+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:21.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:21.165+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:21.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:21.165+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:21.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:21.372+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:21.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:21.372+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:21.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:21.372+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:21.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:21.595+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:21.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:21 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:21.595+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:22.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:21 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:21.578+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:22.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:21 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:21.578+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:22.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:21 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:21.578+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:22.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:21 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:21.578+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:22.327 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:21.939+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:22.327 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:21.939+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:22.327 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:21.939+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:22.327 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:21 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:21.939+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:22.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:22.133+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:22.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:22.133+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:22.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:22.133+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:22.516 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:22 smithi016 podman[98022]: 2024-09-21 03:13:22.233808389 +0000 UTC m=+5.754041567 container died 4e6915b363b383d7512fe75925ca94138171a1ed7e923ab63019dd860dd903e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:22.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:22.325+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:22.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:22.325+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:22.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:22.325+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:22.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:22.628+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:22.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:22 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:22.628+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:22.981 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:22 smithi016 podman[98022]: 2024-09-21 03:13:22.687207396 +0000 UTC m=+6.207440573 container cleanup 4e6915b363b383d7512fe75925ca94138171a1ed7e923ab63019dd860dd903e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:22.981 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:22 smithi016 bash[98022]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6 2024-09-21T03:13:22.981 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:22 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:22.586+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:22.981 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:22 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:22.586+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:22.982 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:22 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:22.586+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:22.982 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:22 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:22.586+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:23.266 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:22 smithi016 podman[98039]: 2024-09-21 03:13:22.980589153 +0000 UTC m=+0.748542961 container remove 4e6915b363b383d7512fe75925ca94138171a1ed7e923ab63019dd860dd903e8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:23.337 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:22.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:23.337 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:22.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:23.337 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:22.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:23.337 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:22.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:23.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:23.171+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:23.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:23.171+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:23.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:23.171+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:23 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:23.536+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:23 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:23.536+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:23 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:23.536+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:23 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:23.536+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.20966963 +0000 UTC m=+0.016928884 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:23.572 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.448248414 +0000 UTC m=+0.255507694 container create 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=) 2024-09-21T03:13:23.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:23.335+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:23.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:23.335+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:23.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:23.335+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:23.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:23.677+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:23.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:23.677+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:23.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:23 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:23.677+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:24.016 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.66741262 +0000 UTC m=+0.474671867 container init 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, org.label-schema.license=GPLv2) 2024-09-21T03:13:24.016 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.671016463 +0000 UTC m=+0.478275701 container start 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:13:24.016 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.783879789 +0000 UTC m=+0.591139041 container attach 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:13:24.016 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:23 smithi016 podman[98125]: 2024-09-21 03:13:23.862148258 +0000 UTC m=+0.669407498 container died 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_CLEAN=True) 2024-09-21T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:23.947+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:23.947+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:23.947+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:24.321 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:23 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:23.947+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:24.390 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.6.service' 2024-09-21T03:13:24.415 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:24 smithi016 podman[98148]: 2024-09-21 03:13:24.370432417 +0000 UTC m=+0.501758770 container remove 358fe9d0985c05bf3b81657515625103f7abe74a38bd708fd24c9195d58ffe3f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-6-deactivate, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:13:24.416 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:24 smithi016 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.6.service: Deactivated successfully. 2024-09-21T03:13:24.416 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:24 smithi016 systemd[1]: Stopped Ceph osd.6 for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T03:13:24.416 INFO:journalctl@ceph.osd.6.smithi016.stdout:Sep 21 03:13:24 smithi016 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.6.service: Consumed 4min 39.336s CPU time. 2024-09-21T03:13:24.433 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:24.434 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-09-21T03:13:24.434 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-09-21T03:13:24.434 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.7 2024-09-21T03:13:24.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:24.125+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:24.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:24.125+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:24.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:24.125+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:24.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.319+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:24.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.319+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:24.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.319+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:24.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 systemd[1]: Stopping Ceph osd.7 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:24.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:24 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:24.490+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:24.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:24 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:24.490+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:24.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:24 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:24.490+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:24.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:24 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:24.490+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:24.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:24.695+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:24.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:24.695+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:24.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:24 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:24.695+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:25.182 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.735+0000 7f82949b8640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.735+0000 7f82949b8640 -1 osd.7 92 *** Got signal Terminated *** 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:24.735+0000 7f82949b8640 -1 osd.7 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:24.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:24.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:24.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:25.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:24 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:24.938+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:25.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:25.145+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:25.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:25.145+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:25.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:25.145+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:25.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:25.310+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:25.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:25.310+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:25.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:25.310+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:25.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:25.452+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:25.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:25.452+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:25.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:25.452+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:25.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:25 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:25.452+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:26.190 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:25.742+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:26.190 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:25.742+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:26.191 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:25 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:25.742+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:26.287 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:25.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:26.287 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:25.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:26.287 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:25.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:26.287 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:25 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:25.951+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:26.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:26.188+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:26.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:26.188+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:26.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:26.188+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:26.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:26.285+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:26.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:26.285+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:26.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:26.285+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:26.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:26 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:26.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:26.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:26 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:26.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:26.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:26 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:26.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:26.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:26 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:26.467+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:27.167 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:26.792+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:27.167 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:26.792+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:27.167 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:26 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:26.792+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:27.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:26.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:27.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:26.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:27.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:26.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:27.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:26.923+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:27.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:27.166+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:27.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:27.166+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:27.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:27.166+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:27.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:27.288+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:27.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:27.288+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:27.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:27.288+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:27.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:27 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:27.425+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:27.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:27 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:27.425+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:27.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:27 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:27.425+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:27.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:27 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:27.425+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:28.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:27.889+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:28.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:27.889+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:28.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:27.889+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:28.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:27.889+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:28.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:27 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:27.889+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:28.194 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:27.826+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:28.194 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:27.826+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:28.194 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:27 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:27.826+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:28.270 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:28.267+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:28.270 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:28.267+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:28.270 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:28.267+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:28.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:28.192+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:28.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:28.192+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:28.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:28.192+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:28.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:28.267+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.813612+0000 front 2024-09-21T03:13:03.813560+0000 (oldest deadline 2024-09-21T03:13:27.313645+0000) 2024-09-21T03:13:28.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:28 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:28.465+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:28.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:28 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:28.465+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:28.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:28 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:28.465+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:28.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:28 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:28.465+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:28.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:28 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:28.465+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:29.151 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:28.779+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:29.151 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:28.779+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:29.151 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:28 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:28.779+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:29.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:28.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:29.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:28.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:29.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:28.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:29.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:28.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:29.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:28 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:28.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:29.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:29.149+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:29.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:29.149+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:29.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:29.149+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:29.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:29.261+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:30.007294+0000 front 2024-09-21T03:12:30.007184+0000 (oldest deadline 2024-09-21T03:12:57.007837+0000) 2024-09-21T03:13:29.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:29.261+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.008308+0000 front 2024-09-21T03:12:37.008343+0000 (oldest deadline 2024-09-21T03:13:14.511282+0000) 2024-09-21T03:13:29.682 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:29.261+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:54.511807+0000 front 2024-09-21T03:12:54.511833+0000 (oldest deadline 2024-09-21T03:13:18.611915+0000) 2024-09-21T03:13:29.683 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7[51603]: 2024-09-21T03:13:29.261+0000 7f828cf9b640 -1 osd.7 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.813612+0000 front 2024-09-21T03:13:03.813560+0000 (oldest deadline 2024-09-21T03:13:27.313645+0000) 2024-09-21T03:13:29.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:29 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:29.484+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:29.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:29 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:29.484+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:29.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:29 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:29.484+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:29.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:29 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:29.484+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:29.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:29 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:29.484+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:30.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:29.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:30.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:29.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:30.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:29.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:30.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:29.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:30.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:29 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:29.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:30.190 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:29.760+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:30.190 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:29.760+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:30.190 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:29.760+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:30.191 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:29 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:29.760+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:30.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:30.189+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:30.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:30.189+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:30.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:30.189+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:30.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:30.189+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:30.644 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:30 smithi018 podman[67254]: 2024-09-21 03:13:30.313868169 +0000 UTC m=+5.679494065 container died 166b3b23c32b6c53c0593d308a5da9c8cfb9cde61d926bca099775d997a3b8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:13:30.644 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:30 smithi018 podman[67254]: 2024-09-21 03:13:30.643359572 +0000 UTC m=+6.008985467 container cleanup 166b3b23c32b6c53c0593d308a5da9c8cfb9cde61d926bca099775d997a3b8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.37.2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:30.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:30.492+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:30.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:30.492+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:30.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:30.492+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:30.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:30.492+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:30.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:30 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:30.492+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:30.932 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:30 smithi018 bash[67254]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7 2024-09-21T03:13:30.932 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:30 smithi018 podman[67273]: 2024-09-21 03:13:30.828014105 +0000 UTC m=+0.512387746 container remove 166b3b23c32b6c53c0593d308a5da9c8cfb9cde61d926bca099775d997a3b8ff (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, org.label-schema.license=GPLv2) 2024-09-21T03:13:30.933 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:30 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:30.872+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:30.933 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:30 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:30.872+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:30.933 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:30 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:30.872+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:30.933 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:30 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:30.872+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:30.933 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:30 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:30.872+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:31.202 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:30.756+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:31.202 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:30.756+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:31.202 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:30.756+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:31.202 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:30 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:30.756+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:31.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:31.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:31.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:31.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:31.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:31.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:31.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:31.200+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:31.506 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.228783862 +0000 UTC m=+0.090887655 container create 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux ) 2024-09-21T03:13:31.507 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.153990267 +0000 UTC m=+0.016094072 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:31.507 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.447699998 +0000 UTC m=+0.309803792 container init 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731) 2024-09-21T03:13:31.507 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.451038003 +0000 UTC m=+0.313141798 container start 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:13:31.507 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.505769916 +0000 UTC m=+0.367873720 container attach 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS) 2024-09-21T03:13:31.923 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:31 smithi018 podman[67361]: 2024-09-21 03:13:31.644503908 +0000 UTC m=+0.506607714 container died 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS) 2024-09-21T03:13:32.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:31.541+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:32.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:31.541+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:32.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:31.541+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:32.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:31.541+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:32.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:31 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:31.541+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:32.181 INFO:journalctl@ceph.osd.7.smithi018.stdout:Sep 21 03:13:32 smithi018 podman[67384]: 2024-09-21 03:13:32.167333553 +0000 UTC m=+0.515932514 container remove 586cfd3d617115da65803172eea01a0a8c6519d0fb6e52b0ee00fa86aca2c86e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-7-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:32.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:31.922+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:32.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:31.922+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:32.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:31.922+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:32.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:31.922+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:32.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:31.922+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:32.186 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.7.service' 2024-09-21T03:13:32.205 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:31.727+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:32.205 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:31.727+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:32.205 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:31.727+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:32.205 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:31 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:31.727+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:32.225 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:32.225 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-09-21T03:13:32.225 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-09-21T03:13:32.225 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.8 2024-09-21T03:13:32.470 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 systemd[1]: Stopping Ceph osd.8 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:32.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:32.204+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:32.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:32.204+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:32.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:32.204+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:32.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:32.204+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:32.970 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.553+0000 7f55a803e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.553+0000 7f55a803e640 -1 osd.8 92 *** Got signal Terminated *** 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.553+0000 7f55a803e640 -1 osd.8 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.700+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.700+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.700+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:32.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:32 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:32.700+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:33.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:32 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:32.518+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:33.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:32 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:32.518+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:33.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:32 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:32.518+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:33.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:32 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:32.518+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:33.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:32 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:32.518+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:33.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:32 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:32.916+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:33.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:32 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:32.916+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:33.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:32 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:32.916+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:33.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:32 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:32.916+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:33.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:32 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:32.916+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:33.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:33.190+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:33.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:33.190+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:33.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:33.190+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:33.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:33.190+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:34.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:33.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:34.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:33.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:34.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:33.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:34.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:33.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:34.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:33 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:33.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:34.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:33.880+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:34.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:33.880+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:34.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:33.880+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:34.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:33.880+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:34.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:33 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:33.880+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:34.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:33.749+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:34.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:33.749+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:34.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:33.749+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:34.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:33 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:33.749+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:34.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:34.223+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:34.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:34.223+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:34.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:34.223+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:34.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:34.223+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:35.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:34 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:34.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:35.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:34 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:34.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:35.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:34 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:34.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:35.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:34 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:34.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:35.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:34 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:34.565+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:35.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:34.867+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:35.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:34.867+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:35.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:34.867+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:35.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:34.867+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:35.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:34.867+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:35.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:34.724+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:35.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:34.724+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:35.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:34.724+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:35.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:34.724+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:35.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:34 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:34.724+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.072102+0000 front 2024-09-21T03:13:09.072084+0000 (oldest deadline 2024-09-21T03:13:34.372188+0000) 2024-09-21T03:13:35.696 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:35.232+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:35.696 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:35.232+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:35.696 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:35.232+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:35.696 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:35.232+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:35.696 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:35.232+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:35.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:35.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:35.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:35.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:35.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:35.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:35.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:35.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:35.971 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:35 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:35.694+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.072102+0000 front 2024-09-21T03:13:09.072084+0000 (oldest deadline 2024-09-21T03:13:34.372188+0000) 2024-09-21T03:13:36.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:35 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:35.614+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:36.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:35 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:35.614+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:36.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:35 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:35.614+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:36.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:35 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:35.614+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:36.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:35 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:35.614+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:36.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:35 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:35.861+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:36.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:35 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:35.861+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:36.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:35 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:35.861+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:36.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:35 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:35.861+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:36.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:35 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:35.861+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:36.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:36.262+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:36.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:36.262+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:36.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:36.262+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:36.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:36.262+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:36.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:36.262+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:37.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:36 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:36.597+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:37.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:36 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:36.863+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:37.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:36 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:36.863+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:37.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:36 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:36.863+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:37.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:36 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:36.863+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:37.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:36 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:36.863+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:37.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:36.743+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:28.767940+0000 front 2024-09-21T03:12:28.767967+0000 (oldest deadline 2024-09-21T03:12:52.168318+0000) 2024-09-21T03:13:37.220 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:36.743+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:33.868967+0000 front 2024-09-21T03:12:33.869031+0000 (oldest deadline 2024-09-21T03:13:11.470422+0000) 2024-09-21T03:13:37.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:36.743+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:56.771222+0000 front 2024-09-21T03:12:56.771162+0000 (oldest deadline 2024-09-21T03:13:22.671159+0000) 2024-09-21T03:13:37.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:36.743+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.371937+0000 front 2024-09-21T03:13:04.371904+0000 (oldest deadline 2024-09-21T03:13:29.071790+0000) 2024-09-21T03:13:37.221 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:36 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8[53001]: 2024-09-21T03:13:36.743+0000 7f55a4629640 -1 osd.8 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.072102+0000 front 2024-09-21T03:13:09.072084+0000 (oldest deadline 2024-09-21T03:13:34.372188+0000) 2024-09-21T03:13:37.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:37 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:37.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:37.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:37 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:37.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:37.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:37 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:37.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:37.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:37 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:37.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:37.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:37 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:37.251+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:38.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:37 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:37.582+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:38.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:38.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:38.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:38.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:38.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:38.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:37 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:37.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:38.386 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 podman[68605]: 2024-09-21 03:13:38.062473464 +0000 UTC m=+5.636676169 container died 87be1a1ec58916811611e7db1daf926f71cc1462c52c202cf49ebe225abc9e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, ceph=True) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 podman[68605]: 2024-09-21 03:13:38.386077474 +0000 UTC m=+5.960280178 container cleanup 87be1a1ec58916811611e7db1daf926f71cc1462c52c202cf49ebe225abc9e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:38.275+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:38.275+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:38.275+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:38.275+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:38.387 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:38.275+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:38.720 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 bash[68605]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8 2024-09-21T03:13:38.720 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 podman[68620]: 2024-09-21 03:13:38.578972202 +0000 UTC m=+0.516515986 container remove 87be1a1ec58916811611e7db1daf926f71cc1462c52c202cf49ebe225abc9e84 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:39.016 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:38 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:38.557+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:39.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:38 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:38.875+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:39.308 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 podman[68711]: 2024-09-21 03:13:38.988074929 +0000 UTC m=+0.111996715 container create 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux ) 2024-09-21T03:13:39.308 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:38 smithi123 podman[68711]: 2024-09-21 03:13:38.891702026 +0000 UTC m=+0.015623827 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:39.308 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:39 smithi123 podman[68711]: 2024-09-21 03:13:39.224359321 +0000 UTC m=+0.348281107 container init 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-09-21T03:13:39.308 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:39 smithi123 podman[68711]: 2024-09-21 03:13:39.227563425 +0000 UTC m=+0.351485222 container start 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:13:39.309 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:39 smithi123 podman[68711]: 2024-09-21 03:13:39.307292675 +0000 UTC m=+0.431214472 container attach 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:39.703 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:39 smithi123 podman[68711]: 2024-09-21 03:13:39.416985843 +0000 UTC m=+0.540907642 container died 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD) 2024-09-21T03:13:39.703 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:39 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:39.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:39.704 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:39 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:39.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:39.704 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:39 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:39.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:39.704 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:39 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:39.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:39.704 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:39 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:39.323+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:39.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:39 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:39.513+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:39.955 INFO:journalctl@ceph.osd.8.smithi123.stdout:Sep 21 03:13:39 smithi123 podman[68734]: 2024-09-21 03:13:39.95236448 +0000 UTC m=+0.529085954 container remove 4924b6cc938ec9ee7bf54021d1c3d70eaac8dda0359fbfb340bc80306aef1fe5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-8-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.37.2) 2024-09-21T03:13:39.976 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.8.service' 2024-09-21T03:13:40.012 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:40.013 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-09-21T03:13:40.013 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-09-21T03:13:40.013 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.9 2024-09-21T03:13:40.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:40.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:40.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:40.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:40.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:40.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:39 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:39.909+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:40.324 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 systemd[1]: Stopping Ceph osd.9 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:40.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:40 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:40.295+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:40.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:40 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:40.295+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:40.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:40 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:40.295+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:40.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:40 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:40.295+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:40.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:40 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:40.295+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.322+0000 7f56b3c87640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.322+0000 7f56b3c87640 -1 osd.9 92 *** Got signal Terminated *** 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.322+0000 7f56b3c87640 -1 osd.9 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:40.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:40 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:40.504+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:41.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:41.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:41.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:41.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:41.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:41.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:40 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:40.911+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:41.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:41.247+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:41.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:41.247+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:41.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:41.247+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:41.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:41.247+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:41.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:41.247+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:41.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:41 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:41.508+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:42.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:42.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:42.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:42.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:42.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:42.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:41 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:41.887+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:42.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:42 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:42.229+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:42.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:42 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:42.229+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:42.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:42 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:42.229+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:42.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:42 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:42.229+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:42.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:42 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:42.229+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:42.765 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:42.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:42.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:42.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:42.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:42.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:42 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:42.460+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:43.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:43.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:43.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:43.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:43.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:43.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:42 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:42.906+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:43.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:43 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:43.182+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:43.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:43 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:43.182+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:43.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:43 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:43.182+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:43.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:43 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:43.182+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:43.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:43 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:43.182+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:43.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:43 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:43.451+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:44.368 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:43 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:43.945+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:44.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:44 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:44.153+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028036+0000 front 2024-09-21T03:12:22.027989+0000 (oldest deadline 2024-09-21T03:12:47.328196+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:27.328545+0000 front 2024-09-21T03:12:27.328829+0000 (oldest deadline 2024-09-21T03:12:51.428666+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.330006+0000 front 2024-09-21T03:12:37.330025+0000 (oldest deadline 2024-09-21T03:13:12.631164+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:57.332038+0000 front 2024-09-21T03:12:57.332074+0000 (oldest deadline 2024-09-21T03:13:21.432237+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:03.633971+0000 front 2024-09-21T03:13:03.634088+0000 (oldest deadline 2024-09-21T03:13:27.733977+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.434868+0000 front 2024-09-21T03:13:12.434939+0000 (oldest deadline 2024-09-21T03:13:35.835509+0000) 2024-09-21T03:13:44.766 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:44 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9[64621]: 2024-09-21T03:13:44.429+0000 7f56b0272640 -1 osd.9 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.036850+0000 front 2024-09-21T03:13:21.036907+0000 (oldest deadline 2024-09-21T03:13:43.936928+0000) 2024-09-21T03:13:45.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:45.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:45.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:45.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:45.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:45.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:45.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:44 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:44.917+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:45.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:45 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:45.177+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:46.265 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:45 smithi016 podman[98227]: 2024-09-21 03:13:45.886792402 +0000 UTC m=+5.668700258 container died 3695efade6e0b721f599bec0866181535211a6c12793f913056a4ae8907c461e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9, ceph=True, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:13:46.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:46.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:45 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:45.942+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:46.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:46.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:46.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:46.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:46.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:46.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:46.174+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:46.717 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:46 smithi016 podman[98227]: 2024-09-21 03:13:46.464846027 +0000 UTC m=+6.246753883 container cleanup 3695efade6e0b721f599bec0866181535211a6c12793f913056a4ae8907c461e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0) 2024-09-21T03:13:46.717 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:46 smithi016 bash[98227]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9 2024-09-21T03:13:47.015 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:46 smithi016 podman[98242]: 2024-09-21 03:13:46.716244303 +0000 UTC m=+0.819016103 container remove 3695efade6e0b721f599bec0866181535211a6c12793f913056a4ae8907c461e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:13:47.304 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:46.94126527 +0000 UTC m=+0.016470957 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:47.304 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:47.167006199 +0000 UTC m=+0.242211871 container create 3cb48a30cfc8e6c3d90a89e353b6f62588c066fc0fbc14d2ba7097c9b8caf873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True) 2024-09-21T03:13:47.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:47.432 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:47.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:47.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:47.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:47.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:47.433 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:46 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:46.936+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:47.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:47.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:47.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:47.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:47.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:47.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:47 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:47.199+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:47.592 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:47.394567689 +0000 UTC m=+0.469773361 container init 3cb48a30cfc8e6c3d90a89e353b6f62588c066fc0fbc14d2ba7097c9b8caf873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9-deactivate, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-09-21T03:13:47.593 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:47.397936911 +0000 UTC m=+0.473142591 container start 3cb48a30cfc8e6c3d90a89e353b6f62588c066fc0fbc14d2ba7097c9b8caf873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9-deactivate, ceph=True, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:13:47.593 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:47.469497197 +0000 UTC m=+0.544702882 container attach 3cb48a30cfc8e6c3d90a89e353b6f62588c066fc0fbc14d2ba7097c9b8caf873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9-deactivate, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:13:47.940 INFO:journalctl@ceph.osd.9.smithi016.stdout:Sep 21 03:13:47 smithi016 podman[98328]: 2024-09-21 03:13:47.592078032 +0000 UTC m=+0.667283721 container died 3cb48a30cfc8e6c3d90a89e353b6f62588c066fc0fbc14d2ba7097c9b8caf873 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-9-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2) 2024-09-21T03:13:48.347 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.9.service' 2024-09-21T03:13:48.392 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:48.393 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-09-21T03:13:48.393 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-09-21T03:13:48.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.10 2024-09-21T03:13:48.420 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:48.421 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:47 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:47.944+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:48.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:48 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:48.193+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:48.682 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 systemd[1]: Stopping Ceph osd.10 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:49.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.693+0000 7f6ef1e1b640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.693+0000 7f6ef1e1b640 -1 osd.10 92 *** Got signal Terminated *** 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.693+0000 7f6ef1e1b640 -1 osd.10 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:49.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:48 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:48.921+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:49.470 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:49.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:49.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:49.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:49.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:49.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:49 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:49.210+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:50.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:50.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:50.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:50.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:50.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:50.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:50.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:49 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:49.920+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:50.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:50.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:50.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:50.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:50.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:50.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:50.245+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:51.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:51.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:51.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:51.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:51.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:51.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:51.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:50 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:50.928+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:51.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:51.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:51 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:51.268+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:52.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:52.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:52.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:52.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:52.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:52.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:52.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:51 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:51.910+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:52.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:52.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:52.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:52.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:52.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:52.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:52 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:52.315+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:53.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:18.636802+0000 front 2024-09-21T03:12:18.636982+0000 (oldest deadline 2024-09-21T03:12:44.537000+0000) 2024-09-21T03:13:53.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.237796+0000 front 2024-09-21T03:12:29.237893+0000 (oldest deadline 2024-09-21T03:12:53.337974+0000) 2024-09-21T03:13:53.182 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:37.939096+0000 front 2024-09-21T03:12:37.939116+0000 (oldest deadline 2024-09-21T03:13:10.443226+0000) 2024-09-21T03:13:53.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6814 osd.3 since back 2024-09-21T03:12:52.144718+0000 front 2024-09-21T03:12:52.144589+0000 (oldest deadline 2024-09-21T03:13:20.945245+0000) 2024-09-21T03:13:53.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:05.447290+0000 front 2024-09-21T03:13:05.447349+0000 (oldest deadline 2024-09-21T03:13:27.747355+0000) 2024-09-21T03:13:53.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:12.448305+0000 front 2024-09-21T03:13:12.448410+0000 (oldest deadline 2024-09-21T03:13:37.148221+0000) 2024-09-21T03:13:53.183 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:52 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10[57966]: 2024-09-21T03:13:52.886+0000 7f6eea3fe640 -1 osd.10 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:19.449245+0000 front 2024-09-21T03:13:19.449510+0000 (oldest deadline 2024-09-21T03:13:44.749056+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:53.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:53 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:53.348+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:54.561 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:54 smithi018 podman[67477]: 2024-09-21 03:13:54.259894085 +0000 UTC m=+5.666212573 container died 81c12857242efe98900fc045d1138dc7a423ffabd25f7c2ee4517da06eb934bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:54.622 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:54.622 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:54.622 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:54.623 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:54.623 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:54.623 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:54.623 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:54 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:54.352+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:54.860 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:54 smithi018 podman[67477]: 2024-09-21 03:13:54.559958554 +0000 UTC m=+5.966277033 container cleanup 81c12857242efe98900fc045d1138dc7a423ffabd25f7c2ee4517da06eb934bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0) 2024-09-21T03:13:54.861 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:54 smithi018 bash[67477]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10 2024-09-21T03:13:54.861 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:54 smithi018 podman[67495]: 2024-09-21 03:13:54.7443505 +0000 UTC m=+0.482331084 container remove 81c12857242efe98900fc045d1138dc7a423ffabd25f7c2ee4517da06eb934bd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731) 2024-09-21T03:13:55.523 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.098628257 +0000 UTC m=+0.015843613 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:13:55.523 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.203449576 +0000 UTC m=+0.120664957 container create 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, CEPH_POINT_RELEASE=) 2024-09-21T03:13:55.523 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.439462022 +0000 UTC m=+0.356677371 container init 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_CLEAN=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:13:55.523 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.442745248 +0000 UTC m=+0.359960588 container start 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:13:55.523 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.522472657 +0000 UTC m=+0.439688009 container attach 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_BRANCH=HEAD, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2) 2024-09-21T03:13:55.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:55.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:55 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:55.331+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:55.932 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:55 smithi018 podman[67582]: 2024-09-21 03:13:55.633234676 +0000 UTC m=+0.550450023 container died 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS) 2024-09-21T03:13:56.226 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.10.service' 2024-09-21T03:13:56.293 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:56 smithi018 podman[67606]: 2024-09-21 03:13:56.200417709 +0000 UTC m=+0.556924362 container remove 33758b9d891c73374afd39736f59459fc1c4fcf0ddba63fcd5b02b7614a144c9 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-10-deactivate, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, io.buildah.version=1.37.2, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-09-21T03:13:56.293 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:56 smithi018 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.10.service: Deactivated successfully. 2024-09-21T03:13:56.293 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:56 smithi018 systemd[1]: Stopped Ceph osd.10 for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T03:13:56.293 INFO:journalctl@ceph.osd.10.smithi018.stdout:Sep 21 03:13:56 smithi018 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.10.service: Consumed 4min 18.146s CPU time. 2024-09-21T03:13:56.311 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:13:56.311 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-09-21T03:13:56.312 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-09-21T03:13:56.312 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.11 2024-09-21T03:13:56.631 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:56.631 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:56.631 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:56.631 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.303+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 systemd[1]: Stopping Ceph osd.11 for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.628+0000 7f05e8e95640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.628+0000 7f05e8e95640 -1 osd.11 92 *** Got signal Terminated *** 2024-09-21T03:13:56.632 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:56 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:56.628+0000 7f05e8e95640 -1 osd.11 92 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-09-21T03:13:57.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:57.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:57 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:57.259+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:58.720 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:58.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:58 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:58.301+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:13:59.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:13:59 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:13:59.280+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2024-09-21T03:13:35.539705+0000 front 2024-09-21T03:13:35.539680+0000 (oldest deadline 2024-09-21T03:13:59.039742+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:14:00.721 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:00 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:00.238+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2024-09-21T03:13:35.539705+0000 front 2024-09-21T03:13:35.539680+0000 (oldest deadline 2024-09-21T03:13:59.039742+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6806 osd.0 since back 2024-09-21T03:12:22.028251+0000 front 2024-09-21T03:12:22.028266+0000 (oldest deadline 2024-09-21T03:12:46.728114+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6804 osd.1 since back 2024-09-21T03:12:29.529317+0000 front 2024-09-21T03:12:29.529425+0000 (oldest deadline 2024-09-21T03:12:54.829315+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6804 osd.2 since back 2024-09-21T03:12:36.530099+0000 front 2024-09-21T03:12:36.530127+0000 (oldest deadline 2024-09-21T03:13:12.831705+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6812 osd.4 since back 2024-09-21T03:13:04.334078+0000 front 2024-09-21T03:13:04.334001+0000 (oldest deadline 2024-09-21T03:13:29.634116+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6812 osd.5 since back 2024-09-21T03:13:09.634403+0000 front 2024-09-21T03:13:09.634450+0000 (oldest deadline 2024-09-21T03:13:34.334470+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.16:6822 osd.6 since back 2024-09-21T03:13:21.236478+0000 front 2024-09-21T03:13:21.236548+0000 (oldest deadline 2024-09-21T03:13:44.136420+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.18:6820 osd.7 since back 2024-09-21T03:13:29.238413+0000 front 2024-09-21T03:13:29.238448+0000 (oldest deadline 2024-09-21T03:13:52.738331+0000) 2024-09-21T03:14:01.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:01 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11[59230]: 2024-09-21T03:14:01.211+0000 7f05e5480640 -1 osd.11 92 heartbeat_check: no reply from 172.21.15.123:6820 osd.8 since back 2024-09-21T03:13:35.539705+0000 front 2024-09-21T03:13:35.539680+0000 (oldest deadline 2024-09-21T03:13:59.039742+0000) 2024-09-21T03:14:02.572 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:02 smithi123 podman[68828]: 2024-09-21 03:14:02.263476998 +0000 UTC m=+5.758629218 container died 6245cab6354cde1a0ef40b4eeeb67bcf41dce3f2de4ca621b7bdae8d6cd5a193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:02.880 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:02 smithi123 podman[68828]: 2024-09-21 03:14:02.645465143 +0000 UTC m=+6.140617356 container cleanup 6245cab6354cde1a0ef40b4eeeb67bcf41dce3f2de4ca621b7bdae8d6cd5a193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-09-21T03:14:02.881 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:02 smithi123 bash[68828]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11 2024-09-21T03:14:03.198 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:02 smithi123 podman[68843]: 2024-09-21 03:14:02.879902068 +0000 UTC m=+0.611778412 container remove 6245cab6354cde1a0ef40b4eeeb67bcf41dce3f2de4ca621b7bdae8d6cd5a193 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-09-21T03:14:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.197345683 +0000 UTC m=+0.112349531 container create ce746eeb98ded2e353ab302f6b31e4253d7a21a521106886c1352b121f15e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11-deactivate, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:14:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.100683335 +0000 UTC m=+0.015687143 image pull ddce307e1f3263fb077b4f3902e15660050631dad4d2823a20ee3f72bf18d977 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262 2024-09-21T03:14:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.424669142 +0000 UTC m=+0.339672941 container init ce746eeb98ded2e353ab302f6b31e4253d7a21a521106886c1352b121f15e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-09-21T03:14:03.471 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.427819194 +0000 UTC m=+0.342823000 container start ce746eeb98ded2e353ab302f6b31e4253d7a21a521106886c1352b121f15e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11-deactivate, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0) 2024-09-21T03:14:03.911 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.516275521 +0000 UTC m=+0.431279343 container attach ce746eeb98ded2e353ab302f6b31e4253d7a21a521106886c1352b121f15e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731) 2024-09-21T03:14:03.912 INFO:journalctl@ceph.osd.11.smithi123.stdout:Sep 21 03:14:03 smithi123 podman[68931]: 2024-09-21 03:14:03.612967085 +0000 UTC m=+0.527970890 container died ce746eeb98ded2e353ab302f6b31e4253d7a21a521106886c1352b121f15e08e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-osd-11-deactivate, ceph=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux ) 2024-09-21T03:14:04.188 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@osd.11.service' 2024-09-21T03:14:04.226 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:04.226 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-09-21T03:14:04.226 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-09-21T03:14:04.226 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.a 2024-09-21T03:14:04.515 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:04 smithi016 systemd[1]: Stopping Ceph mds.a for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:05.016 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a[71280]: 2024-09-21T03:14:04.533+0000 7f02b77a4640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:05.016 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:04 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a[71280]: 2024-09-21T03:14:04.533+0000 7f02b77a4640 -1 mds.a *** got signal Terminated *** 2024-09-21T03:14:07.668 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:07 smithi016 podman[98442]: 2024-09-21 03:14:07.355672794 +0000 UTC m=+2.930607710 container died 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:14:08.015 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:07 smithi016 podman[98442]: 2024-09-21 03:14:07.866987929 +0000 UTC m=+3.441922906 container cleanup 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:14:08.016 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:07 smithi016 bash[98442]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a 2024-09-21T03:14:08.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.a.service' 2024-09-21T03:14:08.377 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:08 smithi016 podman[98455]: 2024-09-21 03:14:08.118675368 +0000 UTC m=+0.758849084 container remove 119aca2f45a12be89da3f2b632c91e26da87f8d66fcf4b8cb37399941cce3ea8 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-a, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23) 2024-09-21T03:14:08.377 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:08 smithi016 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.a.service: Deactivated successfully. 2024-09-21T03:14:08.377 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:08 smithi016 systemd[1]: Stopped Ceph mds.a for ffc7910c-77c3-11ef-baf5-efdc52797490. 2024-09-21T03:14:08.377 INFO:journalctl@ceph.mds.a.smithi016.stdout:Sep 21 03:14:08 smithi016 systemd[1]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.a.service: Consumed 1.128s CPU time. 2024-09-21T03:14:08.390 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:08.391 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-09-21T03:14:08.391 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-09-21T03:14:08.391 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.d 2024-09-21T03:14:08.714 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:08 smithi016 systemd[1]: Stopping Ceph mds.d for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:09.016 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d[70241]: 2024-09-21T03:14:08.712+0000 7f8f23862640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:09.016 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:08 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d[70241]: 2024-09-21T03:14:08.712+0000 7f8f23862640 -1 mds.d *** got signal Terminated *** 2024-09-21T03:14:13.016 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:12 smithi016 podman[98562]: 2024-09-21 03:14:12.715388087 +0000 UTC m=+4.100675755 container died ce70b281ac9810d9f59cf89b6381be053325b4d958d38e71489271c81722bd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240731, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=) 2024-09-21T03:14:13.340 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:13 smithi016 podman[98562]: 2024-09-21 03:14:13.113794646 +0000 UTC m=+4.499082306 container cleanup ce70b281ac9810d9f59cf89b6381be053325b4d958d38e71489271c81722bd03 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2) 2024-09-21T03:14:13.340 INFO:journalctl@ceph.mds.d.smithi016.stdout:Sep 21 03:14:13 smithi016 bash[98562]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-d 2024-09-21T03:14:13.489 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.d.service' 2024-09-21T03:14:13.522 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:13.523 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-09-21T03:14:13.523 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-09-21T03:14:13.523 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.g 2024-09-21T03:14:14.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:13 smithi016 systemd[1]: Stopping Ceph mds.g for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:14.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-g[70975]: 2024-09-21T03:14:13.841+0000 7f4c9c6f9640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:14.016 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:13 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-g[70975]: 2024-09-21T03:14:13.841+0000 7f4c9c6f9640 -1 mds.g *** got signal Terminated *** 2024-09-21T03:14:17.176 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:16 smithi016 podman[98685]: 2024-09-21 03:14:16.869591603 +0000 UTC m=+3.125909736 container died 6bb2634aef8dcbb298a431270ea05fd2a550e5ee3dee7a5547ce9e60d5fc6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-g, io.buildah.version=1.37.2, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, ceph=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:17.515 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:17 smithi016 podman[98685]: 2024-09-21 03:14:17.275193298 +0000 UTC m=+3.531511416 container cleanup 6bb2634aef8dcbb298a431270ea05fd2a550e5ee3dee7a5547ce9e60d5fc6020 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-g, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True) 2024-09-21T03:14:17.516 INFO:journalctl@ceph.mds.g.smithi016.stdout:Sep 21 03:14:17 smithi016 bash[98685]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-g 2024-09-21T03:14:17.669 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.g.service' 2024-09-21T03:14:17.701 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:17.701 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-09-21T03:14:17.701 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-09-21T03:14:17.701 DEBUG:teuthology.orchestra.run.smithi016:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.j 2024-09-21T03:14:18.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:17 smithi016 systemd[1]: Stopping Ceph mds.j for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:18.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j[69861]: 2024-09-21T03:14:18.028+0000 7fa56aba2640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:18.266 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:18 smithi016 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j[69861]: 2024-09-21T03:14:18.028+0000 7fa56aba2640 -1 mds.j *** got signal Terminated *** 2024-09-21T03:14:21.365 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:21 smithi016 podman[98807]: 2024-09-21 03:14:21.090013881 +0000 UTC m=+3.166481364 container died fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:21.723 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:21 smithi016 podman[98807]: 2024-09-21 03:14:21.471161584 +0000 UTC m=+3.547629066 container cleanup fcd4d697c4958b15025a317f0ac9229bd89b1cf2e0e96bd943609874f5e04e7e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j, GIT_CLEAN=True, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:21.723 INFO:journalctl@ceph.mds.j.smithi016.stdout:Sep 21 03:14:21 smithi016 bash[98807]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-j 2024-09-21T03:14:21.843 DEBUG:teuthology.orchestra.run.smithi016:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.j.service' 2024-09-21T03:14:21.879 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:21.879 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-09-21T03:14:21.879 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-09-21T03:14:21.879 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.b 2024-09-21T03:14:22.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:21 smithi018 systemd[1]: Stopping Ceph mds.b for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:22.682 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: 2024-09-21T03:14:22.218+0000 7f0fcc196640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:22.682 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:22 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b[62866]: 2024-09-21T03:14:22.218+0000 7f0fcc196640 -1 mds.b *** got signal Terminated *** 2024-09-21T03:14:25.885 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:25 smithi018 podman[67686]: 2024-09-21 03:14:25.623637276 +0000 UTC m=+3.519233834 container died 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240731, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS) 2024-09-21T03:14:26.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:25 smithi018 podman[67686]: 2024-09-21 03:14:25.983093501 +0000 UTC m=+3.878690099 container cleanup 216605ce03849435b12dabfab4c655f5c4ed749e8e072abcc9017fd6a44ab2d3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-09-21T03:14:26.182 INFO:journalctl@ceph.mds.b.smithi018.stdout:Sep 21 03:14:25 smithi018 bash[67686]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-b 2024-09-21T03:14:26.389 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.b.service' 2024-09-21T03:14:26.425 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:26.425 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-09-21T03:14:26.425 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-09-21T03:14:26.425 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.e 2024-09-21T03:14:26.746 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:26 smithi018 systemd[1]: Stopping Ceph mds.e for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:27.182 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e[62316]: 2024-09-21T03:14:26.744+0000 7f0527306640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:27.182 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:26 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e[62316]: 2024-09-21T03:14:26.744+0000 7f0527306640 -1 mds.e *** got signal Terminated *** 2024-09-21T03:14:31.054 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:30 smithi018 podman[67818]: 2024-09-21 03:14:30.753115974 +0000 UTC m=+4.112054904 container died e339fbc1d0485258d85f9246a0cc949e05d8ccda0861aaa9a96f0c8b1a8f070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, ceph=True, io.buildah.version=1.37.2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:14:31.389 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:31 smithi018 podman[67818]: 2024-09-21 03:14:31.053400363 +0000 UTC m=+4.412339283 container cleanup e339fbc1d0485258d85f9246a0cc949e05d8ccda0861aaa9a96f0c8b1a8f070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-09-21T03:14:31.389 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:31 smithi018 bash[67818]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e 2024-09-21T03:14:31.389 INFO:journalctl@ceph.mds.e.smithi018.stdout:Sep 21 03:14:31 smithi018 podman[67830]: 2024-09-21 03:14:31.25437599 +0000 UTC m=+0.496173091 container remove e339fbc1d0485258d85f9246a0cc949e05d8ccda0861aaa9a96f0c8b1a8f070e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-e, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_BRANCH=HEAD, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux ) 2024-09-21T03:14:31.410 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.e.service' 2024-09-21T03:14:31.443 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:31.443 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-09-21T03:14:31.444 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-09-21T03:14:31.444 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.h 2024-09-21T03:14:31.682 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:31 smithi018 systemd[1]: Stopping Ceph mds.h for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:32.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: 2024-09-21T03:14:31.789+0000 7f678062f640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:32.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:31 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h[62599]: 2024-09-21T03:14:31.789+0000 7f678062f640 -1 mds.h *** got signal Terminated *** 2024-09-21T03:14:33.883 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:33 smithi018 podman[67939]: 2024-09-21 03:14:33.622615591 +0000 UTC m=+1.957872070 container died 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=) 2024-09-21T03:14:34.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:33 smithi018 podman[67939]: 2024-09-21 03:14:33.965058657 +0000 UTC m=+2.300315135 container cleanup 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, org.label-schema.name=CentOS Stream 9 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731) 2024-09-21T03:14:34.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:33 smithi018 bash[67939]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h 2024-09-21T03:14:34.182 INFO:journalctl@ceph.mds.h.smithi018.stdout:Sep 21 03:14:34 smithi018 podman[67951]: 2024-09-21 03:14:34.12452538 +0000 UTC m=+0.499211667 container remove 388109d1357b099934911d25bd561ca7a54355cf57c871419b1d652ad8f76778 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-h, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.37.2, org.label-schema.build-date=20240731, GIT_CLEAN=True) 2024-09-21T03:14:34.344 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.h.service' 2024-09-21T03:14:34.376 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:34.376 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-09-21T03:14:34.376 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-09-21T03:14:34.376 DEBUG:teuthology.orchestra.run.smithi018:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.k 2024-09-21T03:14:34.682 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:34 smithi018 systemd[1]: Stopping Ceph mds.k for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:35.182 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k[62031]: 2024-09-21T03:14:34.705+0000 7f0304284640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:35.182 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:34 smithi018 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k[62031]: 2024-09-21T03:14:34.705+0000 7f0304284640 -1 mds.k *** got signal Terminated *** 2024-09-21T03:14:37.869 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:37 smithi018 podman[68063]: 2024-09-21 03:14:37.604602791 +0000 UTC m=+3.006678505 container died 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 9 Base Image, io.buildah.version=1.37.2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.build-date=20240731, GIT_BRANCH=HEAD, ceph=True) 2024-09-21T03:14:38.182 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:37 smithi018 podman[68063]: 2024-09-21 03:14:37.968617831 +0000 UTC m=+3.370693545 container cleanup 5c241aec2a5b56093241d591cf32a43d9e521449a2a4309f0dfc76da14621a24 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k, ceph=True, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, io.buildah.version=1.37.2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True) 2024-09-21T03:14:38.182 INFO:journalctl@ceph.mds.k.smithi018.stdout:Sep 21 03:14:37 smithi018 bash[68063]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-k 2024-09-21T03:14:38.396 DEBUG:teuthology.orchestra.run.smithi018:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.k.service' 2024-09-21T03:14:38.427 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:38.427 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-09-21T03:14:38.428 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-09-21T03:14:38.428 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.c 2024-09-21T03:14:38.708 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:38 smithi123 systemd[1]: Stopping Ceph mds.c for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:38.968 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c[64325]: 2024-09-21T03:14:38.709+0000 7ffbdae7e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:38.968 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:38 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c[64325]: 2024-09-21T03:14:38.709+0000 7ffbdae7e640 -1 mds.c *** got signal Terminated *** 2024-09-21T03:14:40.577 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:40 smithi123 podman[69034]: 2024-09-21 03:14:40.248912381 +0000 UTC m=+1.631950761 container died 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , ceph=True) 2024-09-21T03:14:40.577 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:40 smithi123 podman[69034]: 2024-09-21 03:14:40.576284112 +0000 UTC m=+1.959322492 container cleanup 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, io.buildah.version=1.37.2, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.name=CentOS Stream 9 Base Image, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=) 2024-09-21T03:14:40.970 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:40 smithi123 bash[69034]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c 2024-09-21T03:14:40.971 INFO:journalctl@ceph.mds.c.smithi123.stdout:Sep 21 03:14:40 smithi123 podman[69058]: 2024-09-21 03:14:40.77749962 +0000 UTC m=+0.525669019 container remove 5dd1953510ee8298204a88079441d93f4f228bb15813db58f16429fafaddf067 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-c, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.license=GPLv2, io.buildah.version=1.37.2, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240731, CEPH_POINT_RELEASE=, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221) 2024-09-21T03:14:40.997 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.c.service' 2024-09-21T03:14:41.033 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:41.033 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-09-21T03:14:41.033 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-09-21T03:14:41.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.f 2024-09-21T03:14:41.356 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:41 smithi123 systemd[1]: Stopping Ceph mds.f for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:41.720 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f[64058]: 2024-09-21T03:14:41.354+0000 7f6cf9571640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:41.720 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:41 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f[64058]: 2024-09-21T03:14:41.354+0000 7f6cf9571640 -1 mds.f *** got signal Terminated *** 2024-09-21T03:14:45.623 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:45 smithi123 podman[69166]: 2024-09-21 03:14:45.370215542 +0000 UTC m=+4.114664559 container died 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 9 Base Image, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, io.buildah.version=1.37.2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-09-21T03:14:45.971 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:45 smithi123 podman[69166]: 2024-09-21 03:14:45.713339382 +0000 UTC m=+4.457788429 container cleanup 094d54b0ac964039712e334cad7885089df9cabb9fa9cd90067514a07eec4425 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, ceph=True, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.vendor=CentOS, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, io.buildah.version=1.37.2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:14:45.971 INFO:journalctl@ceph.mds.f.smithi123.stdout:Sep 21 03:14:45 smithi123 bash[69166]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-f 2024-09-21T03:14:46.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.f.service' 2024-09-21T03:14:46.180 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:46.181 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-09-21T03:14:46.181 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-09-21T03:14:46.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.i 2024-09-21T03:14:46.493 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:46 smithi123 systemd[1]: Stopping Ceph mds.i for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:46.970 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i[64595]: 2024-09-21T03:14:46.491+0000 7ffa0913e640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:46.970 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:46 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i[64595]: 2024-09-21T03:14:46.491+0000 7ffa0913e640 -1 mds.i *** got signal Terminated *** 2024-09-21T03:14:49.609 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:49 smithi123 podman[69287]: 2024-09-21 03:14:49.341259392 +0000 UTC m=+2.947483036 container died 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.37.2, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240731, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-09-21T03:14:49.970 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:49 smithi123 podman[69287]: 2024-09-21 03:14:49.708229074 +0000 UTC m=+3.314452719 container cleanup 24ca7dfb63207c2a3f62fb503238710b26b64f450f2ed9d6b36458009f16f7c2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 9 Base Image, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.37.2, org.label-schema.build-date=20240731) 2024-09-21T03:14:49.971 INFO:journalctl@ceph.mds.i.smithi123.stdout:Sep 21 03:14:49 smithi123 bash[69287]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-i 2024-09-21T03:14:50.126 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.i.service' 2024-09-21T03:14:50.158 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:50.158 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-09-21T03:14:50.159 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-09-21T03:14:50.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.l 2024-09-21T03:14:50.588 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:50 smithi123 systemd[1]: Stopping Ceph mds.l for ffc7910c-77c3-11ef-baf5-efdc52797490... 2024-09-21T03:14:50.971 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l[63791]: 2024-09-21T03:14:50.587+0000 7f6b3d3bf640 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-09-21T03:14:50.971 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:50 smithi123 ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l[63791]: 2024-09-21T03:14:50.587+0000 7f6b3d3bf640 -1 mds.l *** got signal Terminated *** 2024-09-21T03:14:54.880 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:54 smithi123 podman[69407]: 2024-09-21 03:14:54.594449317 +0000 UTC m=+4.219778920 container died 3aead19d935a9387a36da4f79de1a96c542aa008ed6c622e3b48e8af35968a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240731, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.name=CentOS Stream 9 Base Image, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, io.buildah.version=1.37.2) 2024-09-21T03:14:55.221 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:54 smithi123 podman[69407]: 2024-09-21 03:14:54.978472088 +0000 UTC m=+4.603801693 container cleanup 3aead19d935a9387a36da4f79de1a96c542aa008ed6c622e3b48e8af35968a06 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:dd75f9b101bc316b5d0b4eba020bf68e946d78b16cc3e2ed75055abd1b43a262, name=ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-pdonnell-testing-20240920.212106-debug-1843221, ceph=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.37.2, org.label-schema.build-date=20240731, org.label-schema.vendor=CentOS, GIT_COMMIT=92e6e0d02ffae0030ac4c9454206214b98a73a23, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 9 Base Image) 2024-09-21T03:14:55.221 INFO:journalctl@ceph.mds.l.smithi123.stdout:Sep 21 03:14:54 smithi123 bash[69407]: ceph-ffc7910c-77c3-11ef-baf5-efdc52797490-mds-l 2024-09-21T03:14:55.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo pkill -f 'journalctl -f -n 0 -u ceph-ffc7910c-77c3-11ef-baf5-efdc52797490@mds.l.service' 2024-09-21T03:14:55.378 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-09-21T03:14:55.378 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-09-21T03:14:55.378 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:14:55.593 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:14:58.185 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:14:58.395 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:15:00.829 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force --keep-logs 2024-09-21T03:15:01.031 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:15:03.359 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:15:03.393 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:15:03.425 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-09-21T03:15:03.452 INFO:tasks.cephadm:Archiving crash dumps... 2024-09-21T03:15:03.454 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi016/crash 2024-09-21T03:15:03.455 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:15:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:tar: /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:15:03.486 INFO:teuthology.orchestra.run.smithi016.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:15:03.488 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi018/crash 2024-09-21T03:15:03.488 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:15:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:tar: /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:15:03.532 INFO:teuthology.orchestra.run.smithi018.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:15:03.534 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi123/crash 2024-09-21T03:15:03.534 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash -- . 2024-09-21T03:15:03.571 INFO:teuthology.orchestra.run.smithi123.stderr:tar: /var/lib/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/crash: Cannot open: No such file or directory 2024-09-21T03:15:03.571 INFO:teuthology.orchestra.run.smithi123.stderr:tar: Error is not recoverable: exiting now 2024-09-21T03:15:03.573 INFO:tasks.cephadm:Checking cluster log for badness... 2024-09-21T03:15:03.573 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:15:03.620 INFO:teuthology.orchestra.run.smithi016.stdout:2024-09-21T03:03:25.373792+0000 mon.a (mon.0) 1198 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:15:03.620 WARNING:tasks.cephadm:Found errors (ERR|WRN|SEC) in cluster log 2024-09-21T03:15:03.621 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[SEC\]' /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:15:03.660 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[ERR\]' /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:15:03.739 DEBUG:teuthology.orchestra.run.smithi016:> sudo egrep '\[WRN\]' /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v 'fs.*is degraded' | egrep -v 'filesystem is degraded' | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v 'filesystem is offline' | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v 'insufficient standby MDS daemons available' | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'online, but wants' | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'Degraded data redundancy' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-09-21T03:15:03.815 INFO:teuthology.orchestra.run.smithi016.stdout:2024-09-21T03:03:25.373792+0000 mon.a (mon.0) 1198 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT) 2024-09-21T03:15:03.817 INFO:tasks.cephadm:Compressing logs... 2024-09-21T03:15:03.817 DEBUG:teuthology.orchestra.run.smithi016:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:15:03.858 DEBUG:teuthology.orchestra.run.smithi018:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:15:03.861 DEBUG:teuthology.orchestra.run.smithi123:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-09-21T03:15:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.0.log 2024-09-21T03:15:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:: No such file or directory 2024-09-21T03:15:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.9.log 2024-09-21T03:15:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:15:03.882 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.9.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.a.log 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.a.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.a.log 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.g.log 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.a.log: 84.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:15:03.884 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.6.log 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:: No such file or directory 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.i.log 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.c.log 2024-09-21T03:15:03.886 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.i.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:15:03.887 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.g.log: 90.2% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:15:03.887 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.11.log 2024-09-21T03:15:03.888 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.c.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.2.log 2024-09-21T03:15:03.888 INFO:teuthology.orchestra.run.smithi123.stderr: 81.6% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:15:03.888 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.11.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.5.log 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.10.log 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:: No such file or directory 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.1.log 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.7.log 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.10.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log 2024-09-21T03:15:03.889 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.b.log 2024-09-21T03:15:03.893 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.k.log 2024-09-21T03:15:03.893 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.4.log 2024-09-21T03:15:03.893 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.b.log: gzip/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.k.log: -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log 2024-09-21T03:15:03.893 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log 2024-09-21T03:15:03.893 INFO:teuthology.orchestra.run.smithi016.stderr: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.a.log.gz 2024-09-21T03:15:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:15:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.d.log 2024-09-21T03:15:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.x.log 2024-09-21T03:15:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.d.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log: 95.1% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.g.log.gz 2024-09-21T03:15:03.894 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.j.log 2024-09-21T03:15:03.899 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.x.log: 90.8% 86.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:15:03.900 INFO:teuthology.orchestra.run.smithi016.stderr: -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:15:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.3.log 2024-09-21T03:15:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.j.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:15:03.900 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.3.log: 95.1% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.d.log.gz 2024-09-21T03:15:03.900 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log: 90.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:15:03.901 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.y.log 2024-09-21T03:15:03.901 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log: 81.6% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.cephadm.log.gz 2024-09-21T03:15:03.901 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:15:03.901 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.y.log: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.k.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.2.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.5.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log: 90.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.audit.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr: 86.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.l.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.8.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.l.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.z.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.8.log: 95.1% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.i.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.c.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.z.log: 90.9% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.l.log.gz 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.f.log 2024-09-21T03:15:03.903 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:15:03.904 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.f.log: 92.7% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.z.log.gz 2024-09-21T03:15:03.904 INFO:teuthology.orchestra.run.smithi018.stderr: 90.9% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-volume.log.gz 2024-09-21T03:15:03.905 INFO:teuthology.orchestra.run.smithi016.stderr: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.j.log.gz 2024-09-21T03:15:03.905 INFO:teuthology.orchestra.run.smithi123.stderr:/var/log/ceph/cephadm.log: 92.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:15:03.905 INFO:teuthology.orchestra.run.smithi018.stderr: 92.6% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.y.log.gz 2024-09-21T03:15:03.905 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.h.log 2024-09-21T03:15:03.906 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.b.log 2024-09-21T03:15:03.906 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log: /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.h.log: gzip -5 --verbose -- /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.e.log 2024-09-21T03:15:03.907 INFO:teuthology.orchestra.run.smithi123.stderr: 95.1% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.c.log.gz 2024-09-21T03:15:03.908 INFO:teuthology.orchestra.run.smithi123.stderr: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.f.log.gz 2024-09-21T03:15:03.909 INFO:teuthology.orchestra.run.smithi016.stderr:/var/log/ceph/cephadm.log: 93.8% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:15:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.b.log: 86.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph.log.gz 2024-09-21T03:15:03.910 INFO:teuthology.orchestra.run.smithi018.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-09-21T03:15:03.918 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.e.log: 95.0% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.e.log.gz 2024-09-21T03:15:03.920 INFO:teuthology.orchestra.run.smithi018.stderr:/var/log/ceph/cephadm.log: 91.9% -- replaced with /var/log/ceph/cephadm.log.gz 2024-09-21T03:15:04.169 INFO:teuthology.orchestra.run.smithi123.stderr: 91.7% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.c.log.gz 2024-09-21T03:15:04.270 INFO:teuthology.orchestra.run.smithi016.stderr: 90.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mgr.x.log.gz 2024-09-21T03:15:04.328 INFO:teuthology.orchestra.run.smithi018.stderr: 91.7% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.b.log.gz 2024-09-21T03:15:05.174 INFO:teuthology.orchestra.run.smithi016.stderr: 90.2% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mon.a.log.gz 2024-09-21T03:15:07.959 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.9.log.gz 2024-09-21T03:15:08.215 INFO:teuthology.orchestra.run.smithi123.stderr: 93.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.2.log.gz 2024-09-21T03:15:08.257 INFO:teuthology.orchestra.run.smithi123.stderr: 93.2% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.8.log.gz 2024-09-21T03:15:08.812 INFO:teuthology.orchestra.run.smithi123.stderr: 93.3% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.5.log.gz 2024-09-21T03:15:09.196 INFO:teuthology.orchestra.run.smithi018.stderr: 93.6% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.4.log.gz 2024-09-21T03:15:09.431 INFO:teuthology.orchestra.run.smithi018.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.10.log.gz 2024-09-21T03:15:09.667 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.6.log.gz 2024-09-21T03:15:09.807 INFO:teuthology.orchestra.run.smithi123.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.11.log.gz 2024-09-21T03:15:09.809 INFO:teuthology.orchestra.run.smithi123.stderr: 2024-09-21T03:15:09.809 INFO:teuthology.orchestra.run.smithi123.stderr:real 0m5.934s 2024-09-21T03:15:09.809 INFO:teuthology.orchestra.run.smithi123.stderr:user 0m19.061s 2024-09-21T03:15:09.809 INFO:teuthology.orchestra.run.smithi123.stderr:sys 0m0.797s 2024-09-21T03:15:09.833 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.3.log.gz 2024-09-21T03:15:09.847 INFO:teuthology.orchestra.run.smithi018.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.1.log.gz 2024-09-21T03:15:09.911 INFO:teuthology.orchestra.run.smithi016.stderr: 93.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.0.log.gz 2024-09-21T03:15:09.913 INFO:teuthology.orchestra.run.smithi016.stderr: 2024-09-21T03:15:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:real 0m6.043s 2024-09-21T03:15:09.913 INFO:teuthology.orchestra.run.smithi016.stderr:user 0m22.695s 2024-09-21T03:15:09.914 INFO:teuthology.orchestra.run.smithi016.stderr:sys 0m0.831s 2024-09-21T03:15:11.781 INFO:teuthology.orchestra.run.smithi018.stderr: 93.5% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-osd.7.log.gz 2024-09-21T03:15:12.285 INFO:teuthology.orchestra.run.smithi018.stderr: 97.4% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.h.log.gz 2024-09-21T03:16:23.896 INFO:teuthology.orchestra.run.smithi018.stderr: 94.6% -- replaced with /var/log/ceph/ffc7910c-77c3-11ef-baf5-efdc52797490/ceph-mds.b.log.gz 2024-09-21T03:16:23.897 INFO:teuthology.orchestra.run.smithi018.stderr: 2024-09-21T03:16:23.897 INFO:teuthology.orchestra.run.smithi018.stderr:real 1m20.022s 2024-09-21T03:16:23.897 INFO:teuthology.orchestra.run.smithi018.stderr:user 1m49.762s 2024-09-21T03:16:23.898 INFO:teuthology.orchestra.run.smithi018.stderr:sys 0m3.576s 2024-09-21T03:16:23.898 INFO:tasks.cephadm:Archiving logs... 2024-09-21T03:16:23.898 DEBUG:teuthology.misc:Transferring archived files from smithi016:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi016/log 2024-09-21T03:16:23.899 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:16:27.047 DEBUG:teuthology.misc:Transferring archived files from smithi018:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi018/log 2024-09-21T03:16:27.048 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:16:39.421 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi123/log 2024-09-21T03:16:39.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2024-09-21T03:16:42.241 INFO:tasks.cephadm:Removing cluster... 2024-09-21T03:16:42.242 DEBUG:teuthology.orchestra.run.smithi016:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force 2024-09-21T03:16:42.458 INFO:teuthology.orchestra.run.smithi016.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:16:42.756 DEBUG:teuthology.orchestra.run.smithi018:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force 2024-09-21T03:16:42.967 INFO:teuthology.orchestra.run.smithi018.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:16:43.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid ffc7910c-77c3-11ef-baf5-efdc52797490 --force 2024-09-21T03:16:43.547 INFO:teuthology.orchestra.run.smithi123.stdout:Deleting cluster with fsid: ffc7910c-77c3-11ef-baf5-efdc52797490 2024-09-21T03:16:43.826 INFO:tasks.cephadm:Removing cephadm ... 2024-09-21T03:16:43.827 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:16:43.847 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:16:43.864 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf /home/ubuntu/cephtest/cephadm 2024-09-21T03:16:43.880 INFO:tasks.cephadm:Teardown complete 2024-09-21T03:16:43.880 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-09-21T03:16:43.896 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-09-21T03:16:43.896 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:16:43.899 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:16:43.906 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-09-21T03:16:44.009 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-09-21T03:16:44.019 INFO:teuthology.task.clock:Checking final clock skew... 2024-09-21T03:16:44.019 DEBUG:teuthology.orchestra.run.smithi016:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:16:44.022 DEBUG:teuthology.orchestra.run.smithi018:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:16:44.024 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-09-21T03:16:44.037 INFO:teuthology.orchestra.run.smithi018.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:16:44.037 INFO:teuthology.orchestra.run.smithi016.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:16:44.038 INFO:teuthology.orchestra.run.smithi123.stderr:bash: line 1: ntpq: command not found 2024-09-21T03:16:44.041 INFO:teuthology.orchestra.run.smithi018.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:16:44.041 INFO:teuthology.orchestra.run.smithi018.stdout:=============================================================================== 2024-09-21T03:16:44.042 INFO:teuthology.orchestra.run.smithi018.stdout:^* hv01.front.sepia.ceph.com 3 6 377 38 -132us[ -72us] +/- 49ms 2024-09-21T03:16:44.042 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 40 +139us[ +198us] +/- 58ms 2024-09-21T03:16:44.042 INFO:teuthology.orchestra.run.smithi018.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +74us[ +134us] +/- 63ms 2024-09-21T03:16:44.042 INFO:teuthology.orchestra.run.smithi018.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:=============================================================================== 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:^* hv01.front.sepia.ceph.com 3 6 377 41 -139us[ -132us] +/- 49ms 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 39 +122us[ +122us] +/- 58ms 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 39 +53us[ +53us] +/- 63ms 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi016.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-09-21T03:16:44.044 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2024-09-21T03:16:44.045 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv01.front.sepia.ceph.com 3 6 377 40 -133us[ -124us] +/- 49ms 2024-09-21T03:16:44.045 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 38 +137us[ +137us] +/- 58ms 2024-09-21T03:16:44.045 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 40 +77us[ +86us] +/- 63ms 2024-09-21T03:16:44.045 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-09-21T03:16:44.045 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-09-21T03:16:44.058 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-09-21T03:16:44.059 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-09-21T03:16:44.096 DEBUG:teuthology.orchestra.run.smithi016:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:16:44.183 DEBUG:teuthology.orchestra.run.smithi018:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:16:44.252 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-09-21T03:16:44.317 DEBUG:teuthology.orchestra.run.smithi016:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:16:44.348 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:44.349 DEBUG:teuthology.orchestra.run.smithi018:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:16:44.377 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:44.378 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 -e scontext=system_u:system_r:logrotate_t:s0 2024-09-21T03:16:44.405 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:44.405 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-09-21T03:16:44.415 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-09-21T03:16:44.447 INFO:teuthology.task.internal:Duration was 2036.175373 seconds 2024-09-21T03:16:44.447 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-09-21T03:16:44.488 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-09-21T03:16:44.488 DEBUG:teuthology.orchestra.run.smithi016:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:16:44.491 DEBUG:teuthology.orchestra.run.smithi018:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:16:44.493 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-09-21T03:16:44.531 INFO:teuthology.orchestra.run.smithi018.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:16:44.531 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:16:44.532 INFO:teuthology.orchestra.run.smithi016.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-09-21T03:16:45.009 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-09-21T03:16:45.009 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi016.front.sepia.ceph.com 2024-09-21T03:16:45.009 DEBUG:teuthology.orchestra.run.smithi016:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:16:45.034 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi018.front.sepia.ceph.com 2024-09-21T03:16:45.034 DEBUG:teuthology.orchestra.run.smithi018:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:16:45.056 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2024-09-21T03:16:45.057 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-09-21T03:16:45.083 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-09-21T03:16:45.083 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:16:45.086 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:16:45.100 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-09-21T03:16:45.149 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-09-21T03:16:45.149 DEBUG:teuthology.orchestra.run.smithi016:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:16:45.157 DEBUG:teuthology.orchestra.run.smithi018:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:16:45.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-09-21T03:16:46.110 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-09-21T03:16:46.121 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-09-21T03:16:46.122 DEBUG:teuthology.orchestra.run.smithi016:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:16:46.147 DEBUG:teuthology.orchestra.run.smithi018:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:16:46.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-09-21T03:16:46.202 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-09-21T03:16:46.218 DEBUG:teuthology.orchestra.run.smithi016:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.221 DEBUG:teuthology.orchestra.run.smithi018:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.223 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.242 INFO:teuthology.orchestra.run.smithi016.stdout:kernel.core_pattern = core 2024-09-21T03:16:46.246 INFO:teuthology.orchestra.run.smithi018.stdout:kernel.core_pattern = core 2024-09-21T03:16:46.267 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2024-09-21T03:16:46.281 DEBUG:teuthology.orchestra.run.smithi016:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.311 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:46.311 DEBUG:teuthology.orchestra.run.smithi018:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.327 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:46.327 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2024-09-21T03:16:46.342 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-09-21T03:16:46.342 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-09-21T03:16:46.353 INFO:teuthology.task.internal:Transferring archived files... 2024-09-21T03:16:46.354 DEBUG:teuthology.misc:Transferring archived files from smithi016:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi016 2024-09-21T03:16:46.354 DEBUG:teuthology.orchestra.run.smithi016:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:16:46.406 DEBUG:teuthology.misc:Transferring archived files from smithi018:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi018 2024-09-21T03:16:46.406 DEBUG:teuthology.orchestra.run.smithi018:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:16:46.443 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/pdonnell-2024-09-20_23:48:22-fs-wip-pdonnell-testing-20240920.212106-debug-distro-default-smithi/7913439/remote/smithi123 2024-09-21T03:16:46.443 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-09-21T03:16:46.479 INFO:teuthology.task.internal:Removing archive directory... 2024-09-21T03:16:46.479 DEBUG:teuthology.orchestra.run.smithi016:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:16:46.482 DEBUG:teuthology.orchestra.run.smithi018:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:16:46.484 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2024-09-21T03:16:46.529 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-09-21T03:16:46.540 INFO:teuthology.task.internal:Not uploading archives. 2024-09-21T03:16:46.540 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-09-21T03:16:46.571 INFO:teuthology.task.internal:Tidying up after the test... 2024-09-21T03:16:46.571 DEBUG:teuthology.orchestra.run.smithi016:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:16:46.573 DEBUG:teuthology.orchestra.run.smithi018:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:16:46.575 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-09-21T03:16:46.586 INFO:teuthology.orchestra.run.smithi016.stdout: 265453 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:16 /home/ubuntu/cephtest 2024-09-21T03:16:46.589 INFO:teuthology.orchestra.run.smithi018.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:16 /home/ubuntu/cephtest 2024-09-21T03:16:46.590 INFO:teuthology.orchestra.run.smithi123.stdout: 262165 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Sep 21 03:16 /home/ubuntu/cephtest 2024-09-21T03:16:46.591 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-09-21T03:16:46.613 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-09-21T03:16:46.699 INFO:teuthology.run:Summary data: description: fs/workload/{0-centos_9.stream begin/{0-install 1-cephadm 2-logrotate} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v2} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-bitmap omap_limit/10 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/1 standby-replay tasks/{0-subvolume/{with-no-extra-options} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/no 5-quiesce/no 6-workunit/suites/iogen}} duration: 2036.1753733158112 failure_reason: '"2024-09-21T03:03:25.373792+0000 mon.a (mon.0) 1198 : cluster [WRN] Health check failed: 1 OSD(s) experiencing slow operations in BlueStore (BLUESTORE_SLOW_OP_ALERT)" in cluster log' flavor: default owner: scheduled_pdonnell@teuthology success: false 2024-09-21T03:16:46.700 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-09-21T03:16:46.779 INFO:teuthology.run:FAIL